darwin3 darwin:​doc/​phys_pkgs/​exch2.rst	Source navigation Diff markup Identifier search General search
	Version: darwin virus sigma sal Revert   Change

Warning, /doc/phys_pkgs/exch2.rst is written in an unsupported language. File is not indexed.

8679f9097b Jeff* .. _sub_phys_pkg_exch2:
                 
                 exch2: Extended Cubed Sphere Topology
                 -------------------------------------
                 
bf89a37abc Phob* (in directory: :filelink:`pkg/exch2/`)
                 
8679f9097b Jeff* Introduction
                 ++++++++++++
                 
bf89a37abc Phob* The ``exch2`` package extends the original cubed sphere topology
8679f9097b Jeff* configuration to allow more flexible domain decomposition and
                 parallelization.  Cube faces (also called subdomains) may be divided
                 into any number of tiles that divide evenly into the grid point
                 dimensions of the subdomain.  Furthermore, the tiles can run on
                 separate processors individually or in groups, which provides for
                 manual compile-time load balancing across a relatively arbitrary
                 number of processors.
                 
bf89a37abc Phob* The exchange parameters are declared in :filelink:`W2_EXCH_TOPOLOGY.h <pkg/exch2/W2_EXCH2_TOPOLOGY.h>`
                 and assigned in :filelink:`w2_e2setup.F <pkg/exch2/w2_e2setup.F>`. The
8679f9097b Jeff* validity of the cube topology depends on the ``SIZE.h`` file as
                 detailed below.  The default files provided in the release configure a
                 cubed sphere topology of six tiles, one per subdomain, each with
                 32 :math:`\times` 32 grid points, with all tiles running on a single processor.  Both
                 files are generated by Matlab scripts in
bf89a37abc Phob* :filelink:`utils/exch2/matlab-topology-generator`; see :numref:`ssub_exch2_topogen`
8679f9097b Jeff* for details on creating alternate topologies.  Pregenerated examples
                 of these files with alternate topologies are provided under
bf89a37abc Phob* :filelink:`utils/exch2/code-mods` along with the appropriate ``SIZE.h``
8679f9097b Jeff* file for single-processor execution.
                 
                 Invoking exch2
                 ++++++++++++++
                 
                 To use exch2 with the cubed sphere, the following conditions must be
                 met:
                 
bf89a37abc Phob*  - The exch2 package is included when ``genmake2`` is run.  The easiest way to do this is to add the line ``exch2`` to the ``packages.conf`` file -- see Section :ref:`building_code` for general details.
8679f9097b Jeff* 
bf89a37abc Phob*  - An example of :code:`W2_EXCH2_TOPOLOGY.h` and ``w2_e2setup.F`` must reside in a directory containing files symbolically linked by the ``genmake2`` script.  The safest place to put these is the directory indicated in the :code:`-mods=DIR` command line modifier (typically ``../code``), or the build directory.  The default versions of these files reside in :filelink:`pkg/exch2` and are linked automatically if no other versions exist elsewhere in the build path, but they should be left untouched to avoid breaking configurations other than the one you intend to modify.
8679f9097b Jeff* 
                  - Files containing grid parameters, named ``tile00$n$.mitgrid`` where n=(1:6) (one per subdomain), must be in the working directory when the MITgcm executable is run.  These files are provided in the example experiments for cubed sphere configurations with 32 :math:`\times` 32 cube sides -- please contact MITgcm support if you want to generate files for other configurations.
                 
bf89a37abc Phob*  - As always when compiling MITgcm, the file ``SIZE.h`` must be placed where ``genmake2`` will find it.  In particular for exch2, the domain decomposition specified in ``SIZE.h`` must correspond with the particular configuration's topology specified in ``W2_EXCH2_TOPOLOGY.h`` and ``w2_e2setup.F``.  Domain decomposition issues particular to exch2 are addressed in Section :ref:`ssub_exch2_topogen` and :ref:`ssub_exch2mpi` a more general background on the subject relevant to MITgcm is presented in Section :ref:`using_wrapper`.
8679f9097b Jeff* 
                 At the time of this writing the following examples use exch2 and may
                 be used for guidance:
                 
bf89a37abc Phob*  - :filelink:`verification/adjust_nlfs.cs-32x32x1`
                  - :filelink:`verification/adjustment.cs-32x32x1`
                  - :filelink:`verification/aim.5l_cs`
                  - :filelink:`verification/global_ocean.cs32x15`
                  - :filelink:`verification/hs94.cs-32x32x5`
8679f9097b Jeff* 
                 
                 
                 .. _ssub_exch2_topogen:
                 
                 Generating Topology Files for exch2
                 +++++++++++++++++++++++++++++++++++
                 
                 Alternate cubed sphere topologies may be created using the Matlab
bf89a37abc Phob* scripts in :filelink:`utils/exch2/matlab-topology-generator`. Running the
                 m-file :filelink:`driver.m <utils/exch2/matlab-topology-generator/driver.m>`
8679f9097b Jeff* from the Matlab prompt (there are no parameters to pass) generates
bf89a37abc Phob* exch2 topology files ``W2_EXCH2_TOPOLOGY.h`` and
                 ``w2_e2setup.F`` in the working directory and displays a figure of
                 the topology via Matlab -- :numref:`fig_6tile`, :numref:`fig_18tile`, 
                 and :numref:`fig_48tile` are examples of the generated diagrams.  The other 
8679f9097b Jeff* m-files in the directory are
bf89a37abc Phob* subroutines called from :filelink:`driver.m <utils/exch2/matlab-topology-generator/driver.m>` and should not be run ''bare'' except
8679f9097b Jeff* for development purposes.
                 
                 The parameters that determine the dimensions and topology of the
                 generated configuration are :code:`nr`, :code:`nb`, :code:`ng`,
                 :code:`tnx` and :code:`tny`, and all are assigned early in the script.
                 
                 The first three determine the height and width of the subdomains and
                 hence the size of the overall domain.  Each one determines the number
                 of grid points, and therefore the resolution, along the subdomain
                 sides in a ''great circle'' around each the three spatial axes of the cube.  At the time
                 of this writing MITgcm requires these three parameters to be equal,
                 but they provide for future releases  to accomodate different
                 resolutions around the axes to allow subdomains with differing resolutions.
                 
                 The parameters :code:`tnx` and :code:`tny` determine the width and height of
                 the tiles into which the subdomains are decomposed, and must evenly
                 divide the integer assigned to :code:`nr`, :code:`nb` and :code:`ng`.
                 The result is a rectangular tiling of the subdomain.  :numref:`fig_48tile` shows one possible topology for a twenty-four-tile
                 cube, and :numref:`fig_6tile` shows one for six tiles.
                 
                  .. figure:: figs/adjust_cs.*
                     :width: 70%
                     :align: center
                     :alt: cube sphere topology
                     :name: fig_48tile
                 
                     Plot of a cubed sphere topology with a 32 :math:`\times` 192 domain divided into six 32 :math:`\times` 32 subdomains, each of which is divided into eight tiles of width :code:`tnx=16` and height :code:`tny=8` for a total of forty-eight tiles. The colored borders of the subdomains represent the parameters :code:`nr` (red), :code:`ng` (green), and :code:`nb` (blue). This tiling is used in the example verification/adjustment.cs-32x32x1/ with the option (blanklist.txt) to remove the land-only 4 tiles (11,12,13,14) which are filled in red on the plot.
                 
                 
                  .. figure:: figs/polarcap.*
                     :width: 70%
                     :align: center
                     :alt: polar cap topology
                     :name: fig_18tile
                 
                     Plot of a non-square cubed sphere topology with 6 subdomains of different size (nr=90,ng=360,nb=90), divided into one to four tiles each (:code:`tnx=90, tny=90`), resulting in a total of 18 tiles.
                 
                 
                  .. figure:: figs/s6t_32x32.*
                     :width: 70%
                     :align: center
                     :alt: default cubed sphere topology
                     :name: fig_6tile
                 
                     Plot of a cubed sphere topology with a 32 :math:`\times` 192 domain divided into six 32 :math:`\times` 32 subdomains with one tile each (:code:`tnx=32, tny=32`).  This is the default configuration.
                 
                 
                 Tiles can be selected from the topology to be omitted from being
                 allocated memory and processors.  This tuning is useful in ocean
                 modeling for omitting tiles that fall entirely on land.  The tiles
                 omitted are specified in the file blanklist.txt by their tile number in the topology, separated by a newline.
                 
                 
                 
                 .. _ssub_exch2mpi:
                 
                 exch2, SIZE.h, and Multiprocessing
                 ++++++++++++++++++++++++++++++++++
                 
                 
                 Once the topology configuration files are created, each Fortran
                 ``PARAMETER``  in ``SIZE.h`` must be configured to match.
bf89a37abc Phob* :numref:`using_wrapper` povides a general description of domain
                 decomposition within MITgcm and its relation to ``SIZE.h``. The
8679f9097b Jeff* current section specifies constraints that the exch2 package imposes
                 and describes how to enable parallel execution with MPI.
                 
bf89a37abc Phob* As in the general case, the parameters :varlink:`sNx` and
                 :varlink:`sNy` define the size of the individual tiles, and so
                 must be assigned the same respective values as ``tnx`` and
                 ``tny`` in :filelink:`driver.m <utils/exch2/matlab-topology-generator/driver.m>`.
8679f9097b Jeff* 
bf89a37abc Phob* The halo width parameters :varlink:`OLx` and :varlink:`OLy`
8679f9097b Jeff* have no special bearing on exch2 and may be assigned as in the general
bf89a37abc Phob* case. The same holds for :varlink:`Nr`, the number of vertical
8679f9097b Jeff* levels in the model.
                 
bf89a37abc Phob* The parameters :varlink:`nSx`, :varlink:`nSy`, 
                 :varlink:`nPx`, and :varlink:`nPy` relate to the number of
8679f9097b Jeff* tiles and how they are distributed on processors.  When using exch2,
bf89a37abc Phob* the tiles are stored in the ``x`` dimension, and so
                 :varlink:`nSy` =1 in all cases.  Since the tiles as
8679f9097b Jeff* configured by exch2 cannot be split up accross processors without
bf89a37abc Phob* regenerating the topology, :varlink:`nPy` = 1 as well.
8679f9097b Jeff* 
                 The number of tiles MITgcm allocates and how they are distributed
bf89a37abc Phob* between processors depends on :varlink:`nPx` and
                 :varlink:`nSx`.  :varlink:`nSx` is the number of tiles per
                 processor and :varlink:`nPx` is the number of processors.  The
8679f9097b Jeff* total number of tiles in the topology minus those listed in
bf89a37abc Phob* ``blanklist.txt`` must equal :code:`nSx*nPx`.  Note that in order to
8679f9097b Jeff* obtain maximum usage from a given number of processors in some cases,
                 this restriction might entail sharing a processor with a tile that
                 would otherwise be excluded because it is topographically outside of
bf89a37abc Phob* the domain and therefore in ``blanklist.txt``.  For example,
8679f9097b Jeff* suppose you have five processors and a domain decomposition of
                 thirty-six tiles that allows you to exclude seven tiles.  To evenly
                 distribute the remaining twenty-nine tiles among five processors, you
                 would have to run one ''dummy'' tile to make an even six tiles per
bf89a37abc Phob* processor.  Such dummy tiles are *not* listed in
                 ``blanklist.txt``.
8679f9097b Jeff* 
bf89a37abc Phob* The following is an example of ``SIZE.h`` for the six-tile
                 configuration illustrated in :numref:`fig_6tile`
8679f9097b Jeff* running on one processor:
                 
                 ::
                 
                      PARAMETER (
                      &           sNx =  32,
                      &           sNy =  32,
                      &           OLx =   2,
                      &           OLy =   2,
                      &           nSx =   6,
                      &           nSy =   1,
                      &           nPx =   1,
                      &           nPy =   1,
                      &           Nx  = sNx*nSx*nPx,
                      &           Ny  = sNy*nSy*nPy,
                      &           Nr  =   5)
                 
                 The following is an example for the forty-eight-tile topology in
bf89a37abc Phob* :numref:`fig_48tile` running on six processors:
8679f9097b Jeff* 
                 ::
                 
                      PARAMETER (
                      &           sNx =  16,
                      &           sNy =   8,
                      &           OLx =   2,
                      &           OLy =   2,
                      &           nSx =   8,
                      &           nSy =   1,
                      &           nPx =   6,
                      &           nPy =   1,
                      &           Nx  = sNx*nSx*nPx,
                      &           Ny  = sNy*nSy*nPy,
                      &           Nr  =   5)
                 
                 
                 
                 .. _ssub_exch2_key_variables:
                 
                 Key Variables
                 +++++++++++++
                 
                 The descriptions of the variables are divided up into scalars,
                 one-dimensional arrays indexed to the tile number, and two and
                 three-dimensional arrays indexed to tile number and neighboring tile.
                 This division reflects the functionality of these variables: The
                 scalars are common to every part of the topology, the tile-indexed
                 arrays to individual tiles, and the arrays indexed by tile and
                 neighbor to relationships between tiles and their neighbors. 
                 
                 Scalars:
bf89a37abc Phob* ~~~~~~~~
8679f9097b Jeff* 
                 The number of tiles in a particular topology is set with the parameter
bf89a37abc Phob* :varlink:`exch2_nTiles`, and the maximum number of neighbors of any tiles by
                 :varlink:`W2_maxNeighbours`.  These parameters are used for defining the
8679f9097b Jeff* size of the various one and two dimensional arrays that store tile
                 parameters indexed to the tile number and are assigned in the files
bf89a37abc Phob* generated by :filelink:`driver.m <utils/exch2/matlab-topology-generator/driver.m>`.
                 
                 The scalar parameters :code:`exch2_domain_nxt` and :code:`exch2_domain_nyt` express the number
                 of tiles in the ``x`` and ``y`` global indices.  For example, the default
                 setup of six tiles (:numref:`fig_6tile`) has
                 :code:`exch2_domain_nxt=6` and :code:`exch2_domain_nyt=1`.  A
                 topology of forty-eight tiles, eight per subdomain (as in
                 :numref:`fig_48tile`), will have :code:`exch2_domain_nxt=12` and
                 :code:`exch2_domain_nyt=4`.  Note that these parameters express the
8679f9097b Jeff* tile layout in order to allow global data files that are tile-layout-neutral.
                 They have no bearing on the internal storage of the arrays.  The tiles
bf89a37abc Phob* are stored internally in a range from ``bi``  = ``(1:exch2_nTiles)`` in the
                 ``x`` axis, and the  ``y`` axis variable ``bj`` is assumed to 
                 equal 1 throughout the package. 
8679f9097b Jeff* 
                 Arrays indexed to tile number:
bf89a37abc Phob* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
8679f9097b Jeff* 
bf89a37abc Phob* The following arrays are of length :code:`exch2_nTiles` and are indexed to
8679f9097b Jeff* the tile number, which is indicated in the diagrams with the notation
bf89a37abc Phob* ``tn``.  The indices are omitted in the descriptions. 
                 
                 The arrays :varlink:`exch2_tnx` and 
                 :varlink:`exch2_tny` express the ``x`` and ``y`` dimensions of
                 each tile.  At present for each tile :code:`exch2_tnx`=``sNx`` and
                 :code:`exch2_tny` = :code:`sNy`, as assigned in ``SIZE.h`` and described in
                 :numref:`ssub_exch2mpi`. Future releases of MITgcm may allow varying tile
8679f9097b Jeff* sizes. 
                 
bf89a37abc Phob* The arrays :varlink:`exch2_tbasex` and :varlink:`exch2_tbasey` determine the tiles' 
8679f9097b Jeff* Cartesian origin within a subdomain  
                 and locate the edges of different tiles relative to each other.  As
bf89a37abc Phob* an example, in the default six-tile topology (:numref:`fig_6tile`)
                 each index in these arrays is set to 0 since a tile occupies
8679f9097b Jeff* its entire subdomain.  The twenty-four-tile case discussed above will
bf89a37abc Phob* have values of 0 or 16, depending on the quadrant of the
8679f9097b Jeff* tile within the subdomain.  The elements of the arrays
bf89a37abc Phob* :varlink:`exch2_txglobalo` and
                 :varlink:`exch2_txglobalo` are similar to
                 :varlink:`exch2_tbasex` and
                 :varlink:`exch2_tbasey`, but locate the tile edges within the
8679f9097b Jeff* global address space, similar to that used by global output and input
                 files. 
                 
bf89a37abc Phob* The array :varlink:`exch2_myFace` contains the number of 
                 the subdomain of each tile, in a range ``(1:6)`` in the case of the
                 standard cube topology and indicated by **fn** in
                 :numref:`fig_6tile` and
                 :numref:`fig_48tile`. :varlink:`exch2_nNeighbours`
8679f9097b Jeff* contains a count of the neighboring tiles each tile has, and sets 
                 the bounds for looping over neighboring tiles.
bf89a37abc Phob* :varlink:`exch2_tProc` holds the process rank of each
8679f9097b Jeff* tile, and is used in interprocess communication.  
                 
                 
bf89a37abc Phob* The arrays :varlink:`exch2_isWedge`, 
                 :varlink:`exch2_isEedge`,
                 :varlink:`exch2_isSedge`, and
                 :varlink:`exch2_isNedge` are set to 1 if the
                 indexed tile lies on the edge of its subdomain, 0 if
8679f9097b Jeff* not.  The values are used within the topology generator to determine
                 the orientation of neighboring tiles, and to indicate whether a tile
                 lies on the corner of a subdomain.  The latter case requires special
                 exchange and numerical handling for the singularities at the eight
                 corners of the cube. 
                 
                 
                 Arrays Indexed to Tile Number and Neighbor:
bf89a37abc Phob* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
8679f9097b Jeff* 
bf89a37abc Phob* The following arrays have vectors of length :varlink:`W2_maxNeighbours` and
                 :varlink:`exch2_nTiles` and describe the orientations between the the tiles. 
8679f9097b Jeff* 
bf89a37abc Phob* The array :code:`exch2_neighbourId(a,T)` holds the tile number
                 :code:`Tn` for each of the tile number :code:`T`'s neighboring tiles
                 :code:`a`.  The neighbor tiles are indexed
                 :code:`1:exch2_nNeighbours(T)` in the order right to left on the
8679f9097b Jeff* north then south edges, and then top to bottom on the east then west
                 edges.  
                 
bf89a37abc Phob* The :code:`exch2_opposingSend_record(a,T)` array holds the
                 index :code:`b` of the element in :code:`exch2_neighbourId(b,Tn)`
                 that holds the tile number :code:`T`, given
                 :code:`Tn=exch2_neighborId(a,T)`.  In other words,
8679f9097b Jeff* 
                 ::
                 
                    exch2_neighbourId( exch2_opposingSend_record(a,T),
                                       exch2_neighbourId(a,T) ) = T
                 
                 This provides a back-reference from the neighbor tiles. 
                 
bf89a37abc Phob* The arrays :varlink:`exch2_pi` and 
                 :varlink:`exch2_pj` specify the transformations of indices
8679f9097b Jeff* in exchanges between the neighboring tiles.  These transformations are
                 necessary in exchanges between subdomains because a horizontal dimension 
                 in one subdomain 
                 may map to other horizonal dimension in an adjacent subdomain, and
                 may also have its indexing reversed. This swapping arises from the
                 ''folding'' of two-dimensional arrays into a three-dimensional
                 cube. 
                 
bf89a37abc Phob* The dimensions of :code:`exch2_pi(t,N,T)` and :code:`exch2_pj(t,N,T)`
                 are the neighbor ID :code:`N` and the tile number :code:`T` as explained
                 above, plus a vector of length 2 containing transformation
                 factors :code:`t`.  The first element of the transformation vector
8679f9097b Jeff* holds the factor to multiply the index in the same dimension, and the
                 second element holds the the same for the orthogonal dimension.  To
bf89a37abc Phob* clarify, :code:`exch2_pi(1,N,T)` holds the mapping of the ``x`` axis
                 index of tile :code:`T` to the ``x`` axis of tile :code:`T`'s neighbor
                 :code:`N`, and :code:`exch2_pi(2,N,T)` holds the mapping of :code:`T`'s
                 ``x`` index to the neighbor :code:`N`'s ``y`` index. 
8679f9097b Jeff*  
bf89a37abc Phob* One of the two elements of :code:`exch2_pi` or :code:`exch2_pj` for a
                 given tile :code:`T` and neighbor :code:`N` will be 0, reflecting
8679f9097b Jeff* the fact that the two axes are orthogonal.  The other element will be
bf89a37abc Phob* 1 or -1, depending on whether the axes are indexed in
8679f9097b Jeff* the same or opposite directions.  For example, the transform vector of
                 the arrays for all tile neighbors on the same subdomain will be
bf89a37abc Phob* ``(1,0)``, since all tiles on the same subdomain are oriented
8679f9097b Jeff* identically.  An axis that corresponds to the orthogonal dimension
                 with the same index direction in a particular tile-neighbor
bf89a37abc Phob* orientation will have ``(0,1)``.  Those with the opposite index
                 direction will have ``(0,-1)`` in order to reverse the ordering. 
8679f9097b Jeff* 
bf89a37abc Phob* The arrays :varlink:`exch2_oi`, 
                 :varlink:`exch2_oj`, :varlink:`exch2_oi_f`, and
                 :varlink:`exch2_oj_f` are indexed to tile number and
8679f9097b Jeff* neighbor and specify the relative offset within the subdomain of the
bf89a37abc Phob* array index of a variable going from a neighboring tile :code:`N` to a
                 local tile :code:`T`.  Consider :code:`T=1` in the six-tile topology
                 (:numref:`fig_6tile`), where
8679f9097b Jeff* 
                 ::
                 
                        exch2_oi(1,1)=33
                        exch2_oi(2,1)=0
                        exch2_oi(3,1)=32
                        exch2_oi(4,1)=-32
                 
                 
bf89a37abc Phob* The simplest case is :code:`exch2_oi(2,1)`, the southern neighbor,
                 which is :code:`Tn=6`.  The axes of :code:`T` and :code:`Tn` have the
                 same orientation and their ``x`` axes have the same origin, and so an
                 exchange between the two requires no changes to the ``x`` index.  For
                 the western neighbor (:code:`Tn=5`), :code:`code_oi(3,1)=32` since the
                 :code:`x=0` vector on :code:`T` corresponds to the :code:`y=32` vector on
                 :code:`Tn`.  The eastern edge of :code:`T` shows the reverse case
                 (:code:`exch2_oi(4,1)=-32)`), where :code:`x=32` on :code:`T` exchanges
                 with :code:`x=0` on :code:`Tn=2`. 
                 
                 The most interesting case, where :code:`exch2_oi(1,1)=33` and
                 :code:`Tn=3`, involves a reversal of indices.  As in every case, the
                 offset :code:`exch2_oi` is added to the original ``x`` index of :code:`T`
                 multiplied by the transformation factor :code:`exch2_pi(t,N,T)`.  Here
                 :code:`exch2_pi(1,1,1)=0` since the ``x`` axis of :code:`T` is orthogonal
                 to the ``x`` axis of :code:`Tn`.  :code:`exch2_pi(2,1,1)=-1` since the
                 ``x`` axis of :code:`T` corresponds to the ``y`` axis of :code:`Tn`, but the
8679f9097b Jeff* index is reversed.  The result is that the index of the northern edge
bf89a37abc Phob* of :code:`T`, which runs :code:`(1:32)`, is transformed to
                 :code:`(-1:-32)`. :code:`exch2_oi(1,1)` is then added to this range to
                 get back :code:`(32:1)` -- the index of the ``y`` axis of :code:`Tn`
                 relative to :code:`T`.  This transformation may seem overly convoluted
8679f9097b Jeff* for the six-tile case, but it is necessary to provide a general
                 solution for various topologies. 
                 
                 
                 
bf89a37abc Phob* Finally, :varlink:`exch2_itlo_c`, 
                 :varlink:`exch2_ithi_c`,
                 :varlink:`exch2_jtlo_c` and
                 :varlink:`exch2_jthi_c` hold the location and index
                 bounds of the edge segment of the neighbor tile :code:`N`'s subdomain
                 that gets exchanged with the local tile :code:`T`.  To take the example
                 of tile :code:`T=2` in the forty-eight-tile topology
                 (:numref:`fig_48tile`): 
8679f9097b Jeff* 
                 ::
                 
                        exch2_itlo_c(4,2)=17
                        exch2_ithi_c(4,2)=17
                        exch2_jtlo_c(4,2)=0
                        exch2_jthi_c(4,2)=33
                 
                  
bf89a37abc Phob* Here :code:`N=4`, indicating the western neighbor, which is
                 :code:`Tn=1`.  :code:`Tn` resides on the same subdomain as :code:`T`, so
                 the tiles have the same orientation and the same ``x`` and ``y`` axes.
                 The ``x`` axis is orthogonal to the western edge and the tile is 16
                 points wide, so :code:`exch2_itlo_c` and :code:`exch2_ithi_c`
                 indicate the column beyond :code:`Tn`'s eastern edge, in that tile's
8679f9097b Jeff* halo region. Since the border of the tiles extends through the entire
bf89a37abc Phob* height of the subdomain, the ``y`` axis bounds :code:`exch2_jtlo_c` to
                 :code:`exch2_jthi_c` cover the height of :code:`(1:32)`, plus 1 in
8679f9097b Jeff* either direction to cover part of the halo. 
                 
bf89a37abc Phob* For the north edge of the same tile :code:`T=2` where :code:`N=1` and 
                 the neighbor tile is :code:`Tn=5`:
8679f9097b Jeff* 
                 ::
                 
                        exch2_itlo_c(1,2)=0
                        exch2_ithi_c(1,2)=0
                        exch2_jtlo_c(1,2)=0
                        exch2_jthi_c(1,2)=17
                 
                 
                  
bf89a37abc Phob* :code:`T`'s northern edge is parallel to the ``x`` axis, but since
                 :code:`Tn`'s ``y`` axis corresponds to :code:`T`'s ``x`` axis, :code:`T`'s
                 northern edge exchanges with :code:`Tn`'s western edge.  The western
                 edge of the tiles corresponds to the lower bound of the ``x`` axis, so
                 :code:`exch2_itlo_c` and :code:`exch2_ithi_c` are 0, in the 
                 western halo region of :code:`Tn`. The range of
                 :code:`exch2_jtlo_c` and :code:`exch2_jthi_c` correspond to the
                 width of :code:`T`'s northern edge, expanded by one into the halo. 
8679f9097b Jeff* 
                 
                 Key Routines
                 ++++++++++++
                 
                 Most of the subroutines particular to exch2 handle the exchanges
                 themselves and are of the same format as those described in
bf89a37abc Phob* :ref:`cubed_sphere_comm`.  Like the original routines, they are written as
                 templates which the local Makefile converts from :code:`RX` into 
                 :code:`RL` and :code:`RS` forms. 
                 
                 The interfaces with the core model subroutines are 
                 :code:`EXCH_UV_XY_RX`, :code:`EXCH_UV_XYZ_RX` and
                 :code:`EXCH_XY_RX`.  They override the standard exchange routines
                 when :code:`genmake2` is run with :code:`exch2` option.  They in turn
                 call the local exch2 subroutines :code:`EXCH2_UV_XY_RX` and
                 :code:`EXCH2_UV_XYZ_RX` for two and three-dimensional vector
                 quantities, and :code:`EXCH2_XY_RX` and :code:`EXCH2_XYZ_RX` for two
8679f9097b Jeff* and three-dimensional scalar quantities.  These subroutines set the
bf89a37abc Phob* dimensions of the area to be exchanged, call :code:`EXCH2_RX1_CUBE`
                 for scalars and :code:`EXCH2_RX2_CUBE` for vectors, and then handle
8679f9097b Jeff* the singularities at the cube corners. 
                 
bf89a37abc Phob* The separate scalar and vector forms of :code:`EXCH2_RX1_CUBE` and
                 :code:`EXCH2_RX2_CUBE` reflect that the vector-handling subroutine
8679f9097b Jeff* needs to pass both the $u$ and $v$ components of the physical vectors.
                 This swapping arises from the topological folding discussed above, where the
bf89a37abc Phob* ``x`` and ``y`` axes get swapped in some cases, and is not an
8679f9097b Jeff* issue with the scalar case. These subroutines call
bf89a37abc Phob* :code:`EXCH2_SEND_RX1` and :code:`EXCH2_SEND_RX2`, which do most of
8679f9097b Jeff* the work using the variables discussed above. 
bf89a37abc Phob*     
8679f9097b Jeff* .. _ssub_exch2_experiments:
                 
                 Experiments and tutorials that use exch2
                 ++++++++++++++++++++++++++++++++++++++++
                 
bf89a37abc Phob*  - Held Suarez tutorial, in :filelink:`verification/tutorial_held_suarez_cs` verification directory.

This page was automatically generated from https://github.com/darwinproject/darwin3 by the 2.3.7-MITgcm-0.1 LXR engine. The LXR team