Transfers a user vector to a remote task, obtaining the target address on the remote task from a user-specified header handler.
Availability Library (liblapi_r.a)
#include <lapi.h>
typedef void (compl_hndlr_t) (hndl, user_info);
lapi_handle_t *hndl; /* the LAPI handle passed in from LAPI_Amsendv */
void *user_info; /* the buffer (user_info) pointer passed in */
/* from vhdr_hndlr (void *(vhdr_hndlr_t)) */
typedef lapi_vec_t *(vhdr_hndlr_t) (hndl, uhdr, uhdr_len, len_vec, comp_h, uinfo);
lapi_handle_t *hndl; /* pointer to the LAPI handle passed in from LAPI_Amsendv */
void *uhdr; /* uhdr passed in from LAPI_Amsendv */
uint *uhdr_len; /* uhdr_len passed in from LAPI_Amsendv */
ulong *len_vec[ ]; /* vector of lengths passed in LAPI_Amsendv */
compl_hndlr_t **comp_h; /* function address of completion handler */
/* (void (compl_hndlr_t)) that needs to be */
/* filled out by this header handler function */
void **user_info; /* pointer to the parameter to be passed */
/* in to the completion handler */
int LAPI_Amsendv(hndl, tgt, hdr_hdl, uhdr, uhdr_len, org_vec,
tgt_cntr, org_cntr, cmpl_cntr);
lapi_handle_t hndl;
uint tgt;
void *hdr_hdl;
void *uhdr;
uint uhdr_len;
lapi_vec_t *org_vec;
lapi_cntr_t *tgt_cntr;
lapi_cntr_t *org_cntr;
lapi_cntr_t *cmpl_cntr; include 'lapif.h'
INTEGER SUBROUTINE COMPL_H (hndl, user_info)
INTEGER hndl
INTEGER user_info(*)
INTEGER FUNCTION VHDR_HDL (hndl, uhdr, uhdr_len, len_vec, comp_h, user_info)
INTEGER hndl
INTEGER uhdr
INTEGER uhdr_len
INTEGER (KIND=LAPI_LONG_TYPE) :: len_vec
EXTERNAL INTEGER FUNCTION comp_h
TYPE (LAPI_ADDR_T) :: user_info
LAPI_AMSENDV(hndl, tgt, hdr_hdl, uhdr, uhdr_len, org_vec,
tgt_cntr, org_cntr, cmpl_cntr, ierror)
INTEGER hndl
INTEGER tgt
EXTERNAL INTEGER FUNCTION hdr_hdl
INTEGER uhdr
INTEGER uhdr_len
TYPE (LAPI_VEC_T) :: org_vec
INTEGER (KIND=LAPI_ADDR_TYPE) :: tgt_cntr
TYPE (LAPI_CNTR_T) :: org_cntr
TYPE (LAPI_CNTR_T) :: cmpl_cntr
INTEGER ierror
Type of call: point-to-point communication (non-blocking)
LAPI_Amsendv is the vector-based version of the LAPI_Amsend call. You can use it to specify multi-dimensional and non-contiguous descriptions of the data to transfer. Whereas regular LAPI calls allow the specification of a single data buffer address and length, the vector versions allow the specification of a vector of address and length combinations. Additional information is allowed in the data description on the origin task and the target task.
Use this subroutine to transfer a vector of data to a target task, when you want a handler to run on the target task before message delivery begins or after message delivery completes.
To use LAPI_Amsendv, you must provide a header handler, which returns the address of the target vector description that LAPI uses to write the data that is described by the origin vector. The header handler is used to specify the address of the vector description for writing the data, which eliminates the need to know the description on the origin task when the subroutine is called. The header handler is called upon arrival of the first data packet at the target.
Optionally, you can also provide a completion handler. The header handler provides additional information to LAPI about the message delivery, such as the completion handler. You can also specify a completion handler parameter from within the header handler. LAPI passes the information to the completion handler at execution.
With the exception of the address that is returned by the completion handler, the use of counters, header handlers, and completion handlers in LAPI_Amsendv is identical to that of LAPI_Amsend. In both cases, the user header handler returns information that LAPI uses for writing at the target. See LAPI_Amsend for more information. This section presents information that is specific to the vector version of the call (LAPI_Amsendv).
typedef struct {
lapi_vectype_t vec_type;
uint num_vecs;
void **info;
ulong *len;
} lapi_vec_t;vec_type = LAPI_GEN_IOVECTOR
num_vecs = 3
info = {addr_0, addr_1, addr_2}
len = {len_0, len_1, len_2}Recall that vector transfers require an origin and target vector. For LAPI_Amsendv calls, the origin vector is passed to the API call on the origin task. The address of the target vector is returned by the header handler.
For transfers of type LAPI_GEN_GENERIC, the target vector description must also have type LAPI_GEN_GENERIC. The contents of the info and len arrays are unrestricted in the generic case; the number of vectors and the length of vectors on the origin and target do not need to match. In this case, LAPI transfers a given number of bytes in noncontiguous buffers specified by the origin vector to a set of noncontiguous buffers specified by the target vector.
Origin_vector: {
num_vecs = 3;
info = {orgaddr_0, orgaddr_1, orgaddr_2};
len = {5, 10, 5}
}
Target_vector: {
num_vecs = 4;
info = {tgtaddr_0, tgtaddr_1, tgtaddr_2, tgtaddr_3};
len = {12, 2, 4, 2}
}Origin_vector: {
num_vecs = 1;
info = {orgaddr_0};
len = {20}
}
Target_vector: {
num_vecs = 2;
info = {tgtaddr_0, tgtaddr_1};
len = {5, 10}
}Strided vector transfers
Origin_vector {
num_vecs = 3;
info = {orgaddr, 5, 8}
}Call details
As mentioned above, counter and handler behavior in LAPI_Amsendv is nearly identical to that of LAPI_Amsend. A short summary of that behavior is provided here. See the LAPI_Amsend description for full details.
This is a non-blocking call. The calling task cannot change the uhdr (origin header) and org_vec data until completion at the origin is signaled by the org_cntr being incremented. The calling task cannot assume that the org_vec structure can be changed before the origin counter is incremented. The structure (of type lapi_vec_t) that is returned by the header handler cannot be modified before the target counter has been incremented. Also, if a completion handler is specified, it may execute asynchronously, and can only be assumed to have completed after the target counter increments (on the target) or the completion counter increments (at the origin).
The length of the user-specified header (uhdr_len) is constrained by the implementation-specified maximum value MAX_UHDR_SZ. uhdr_len must be a multiple of the processor's doubleword size. To get the best bandwidth, uhdr_len should be as small as possible.
LAPI does not check for any overlapping regions among vectors either at the origin or the target. If the overlapping regions exist on the target side, the contents of the target buffer are undefined after the operation.
/* header handler routine to execute on target task */
lapi_vec_t *hdr_hndlr(lapi_handle_t *handle, void *uhdr, uint *uhdr_len,
ulong *len_vec[ ], compl_hndlr_t **completion_handler,
void **user_info)
{
/* set completion handler pointer and other info */
/* set up the vector to return to LAPI */
/* for a LAPI_GEN_IOVECTOR: num_vecs, vec_type, and len must all have */
/* the same values as the origin vector. The info array should */
/* contain the buffer addresses for LAPI to write the data */
vec->num_vecs = NUM_VECS;
vec->vec_type = LAPI_GEN_IOVECTOR;
vec->len = (unsigned long *)malloc(NUM_VECS*sizeof(unsigned long));
vec->info = (void **) malloc(NUM_VECS*sizeof(void *));
for( i=0; i < NUM_VECS; i++ ) {
vec->info[i] = (void *) &data_buffer[i];
vec->len[i] = (unsigned long)(sizeof(int));
}
return vec;
}
{
.
.
.
void *hdr_hndlr_list[NUM_TASKS]; /* table of remote header handlers */
lapi_vec_t *vec; /* data for data transfer */
vec->num_vecs = NUM_VECS;
vec->vec_type = LAPI_GEN_IOVECTOR;
vec->len = (unsigned long *) malloc(NUM_VECS*sizeof(unsigned long));
vec->info = (void **) malloc(NUM_VECS*sizeof(void *));
/* each vec->info[i] gets a base address */
/* each vec->len[i] gets the number of bytes to transfer from vec->info[i] */
LAPI_Amsendv(hndl, tgt, (void *) hdr_hdl_list[buddy], NULL, 0, vec,
tgt_cntr, org_cntr, cmpl_cntr);
/* data will be copied as follows: */
/* len[0] bytes of data starting from address info[0] */
/* len[1] bytes of data starting from address info[1] */
.
.
.
/* len[NUM_VECS-1] bytes of data starting from address info[NUM_VECS-1] */
}/* header handler routine to execute on target task */
lapi_vec_t *hdr_hndlr(lapi_handle_t *handle, void *uhdr, uint *uhdr_len,
ulong *len_vec[ ], compl_hndlr_t **completion_handler,
void **user_info)
{
int block_size; /* block size */
int data_size; /* stride */
.
.
.
vec->num_vecs = NUM_VECS; /* NUM_VECS = number of vectors to transfer */
/* must match that of the origin vector */
vec->vec_type = LAPI_GEN_STRIDED_XFER; /* same as origin vector */
/* see comments in origin vector setup for a description of how data */
/* will be copied based on these settings. */
vec->info[0] = buffer_address; /* starting address for data copy */
vec->info[1] = block_size; /* bytes of data to copy */
vec->info[2] = stride; /* distance from copy block to copy block */
.
.
.
return vec;
}
{
.
.
.
lapi_vec_t *vec; /* data for data transfer */
vec->num_vecs = NUM_VECS; /* NUM_VECS = number of vectors to transfer */
/* must match that of the target vector */
vec->vec_type = LAPI_GEN_STRIDED_XFER; /* same as target vector */
vec->info[0] = buffer_address; /* starting address for data copy */
vec->info[1] = block_size; /* bytes of data to copy */
vec->info[2] = stride; /* distance from copy block to copy block */
/* data will be copied as follows: */
/* block_size bytes will be copied from buffer_address */
/* block_size bytes will be copied from buffer_address+stride */
/* block_size bytes will be copied from buffer_address+(2*stride) */
/* block_size bytes will be copied from buffer_address+(3*stride) */
.
.
.
/* block_size bytes will be copied from buffer_address+((NUM_VECS-1)*stride) */
.
.
.
/* if uhdr isn't used, uhdr should be NULL and uhdr_len should be 0 */
/* tgt_cntr, org_cntr and cmpl_cntr can all be NULL */
LAPI_Amsendv(hndl, tgt, (void *) hdr_hdl_list[buddy], uhdr, uhdr_len,
vec, tgt_cntr, org_cntr, cmpl_cntr);
.
.
.
}For complete examples, see the sample programs shipped with LAPI.