Each node initialises its own particles to to random positions:
srand48((long) my_rank);
for (i = 0; i < particle_number; i++) {
particles[my_offset + i].x = drand48();
particles[my_offset + i].y = drand48();
particles[my_offset + i].z = drand48();
particles[my_offset + i].mass = 1.0;
}
Observe that prior to using drand48 each processor seeds the
random number generator with its own rank number within the
MPI_COMM_WORLD communicator. We have to do that because
the function drand48 is random but entirely deterministic.
Without seeding it, all processors would deliver
a particle_number of identical particles. Beware
that Argonne program
nbody.c, which this file is based on, does not use seeding.
Once every node initialised its own segment, the nodes exchange their particles with all other nodes:
MPI_Allgatherv ( particles + my_offset, particle_number, particle_type, particles, counts, displacements, particle_type, MPI_COMM_WORLD );
This new operation is related to MPI_Allgather, but is
somewhat more complex. MPI_Allgather assumed that
all processes would send chunks of data of identical length.
MPI_Allgatherv, on the other hand, can collect chunks of data of
different length from several nodes. In our case this is an overkill,
because we still send chunks of data of identical length, but we
could distribute particles unequally amongst processors,
for example, in order to balance their unequal performance.
MPI_Allgatherv takes from each processor a batch of data
described by its first three arguments: a buffer pointer
(particles + my_offset), number of
items (particle_number), and MPI type (particle_type).
The data is assembled into an array given
by its fourth argument (particles).
The number of data received from each
processor is specified by the array given as a fifth argument
(counts),
and the places where the data should be inserted in the destination
buffer is given again by an array which is the sixth argument
(displacements).
MPI_Allgatherv also has its close relative MPI_Gatherv,
which does much the same job as MPI_Allgatherv, but the gathered
data is delivered only to the root process.
The result of this operation is that by now all nodes know everything there is to know about all particles. Hence the evaluation of forces can commence.