Current Page: NeXtMidas User's Guide X-Midas Interoperability NeXtOpt
| ### Using NeXtMidas as an X-Midas Option Tree (NeXtOpt) NeXtMidas may be used as an X-Midas option tree. This enables an X-Midas macro to call NeXtMidas primitives. Requirements Current Capabilities Building Running Using X-Midas Results from NeXtMidas Using X-Midas Pipes from NeXtMidas Using X-Midas Messages from NeXtMidas Using X-Midas AUX List from NeXtMidas Concept and Design Debugging Top Ten Things To Remember #### Requirements Install `libXt-devel` (RHEL/CentOS) or appropriate package that provides required development headers for X11/Intrinsic.h. In order to build NeXtMidas as an X-Midas option tree you will need to use three compilers: A Fortran compiler (for the host primitives) A C compiler (for the Java Native Interface, JNI) A Java compiler (for NeXtMidas) The following table shows the supported compilers and X-Midas versions. Other combinations of compliers for X-Midas 3.8.X and 4.2.X are likely work out-of-the-box, they are red here because (as of the time of this writing) no one has tried them yet. It is possible that a few minor changes will be necessary before X-Midas 3.6.5 and 3.7.1 are supported. Any help is welcome in getting this matrix filled in.
| Fortran Compilers | C Compilers
Promula | Intel (ifort) | PGF | GCC | Intel (icc)
6.5 | 9.1 | 10.1 | 5.2 | 6.0 | 8.0 | 3.2.X | 3.4.X | 4.1.X | 10.1
X-Midas 4.8.X | Yes(1) | Yes | Yes | Yes(1) | Yes(1) | Yes | Yes | Yes | Yes | Yes
X-Midas 4.6.X | Yes(1) | Yes | Yes | Yes(1) | Yes(1) | Yes | Yes | Yes | Yes | Yes
The below X-Midas version are no longer supported.
X-Midas 4.4.X | Yes(1) | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes
X-Midas 4.2.X | Yes | No | No | Yes(1) | Yes(1) | No | Yes | Yes(1) | No | No
X-Midas 3.8.3 | Yes | No | No | No | No | No | Yes | No | No | No
X-Midas 3.7.1 | No | No | No | No | No | No | No | No | No | No
X-Midas 3.6.5 | No | No | No | No | No | No | No | No | No | No
All versions require Java 7 or later
(1) User tested.
- #### Current Capabilities As of this writing, NeXtMidas primitives run from the X-Midas shell can: Read from X-Midas pipes.
  Limitations: | NeXtMidas supports dynamic allocation and connection of pipes. This is not supported when using X-Midas pipes. The NeXtMidas primitive must open an X-Midas pipe inside its' `open()` method, otherwise the primitive will not see the start of the pipe. On the NeXtMidas side the pipe name must be prefixed with `xmpipe:`.
- Create and write to X-Midas pipes.
  Limitations: | NeXtMidas supports dynamic allocation and connection of pipes. This is not supported when using X-Midas pipes. NeXtMidas has no control over the size of the pipes (pipe size is set by X-Midas). When a NeXtMidas primitive is creating an X-Midas pipe, it must set the file size (note that this is not the same as pipe size) before it opens the pipe. The NeXtMidas primitive must open an X-Midas pipe inside its' `open()` method. The X-Midas `PIPE` or `XPIPE` must have the `/NP=` set to a value large enough to account for the pipes created by NeXtMidas primitives. On the NeXtMidas side the pipe name must be prefixed with `xmpipe:`.
- Read an X-Midas pipe created by another NeXtMidas primitive. (This will, however, be slower than just using NeXtMidas pipes.) Read/write results from/to the X-Midas results table.
  Limitations: | NeXtMidas supports result groups or collections called TABLES and can have result objects. These features are not supported in the X-Midas results table.
- Read and set the AUX list to find and create files. NeXtMidas has all capabilites X-Midas has to dynamically read and set the AUX list.
  Limitations: | NeXtMidas supports AUX names as strings, but X-Midas only allows numbers. NeXtMidas supports URLs but X-Midas only allows local directories.
- Send messages from X-Midas to NeXtMidas and send messages from NeXtMidas to X-Midas.
  Limitations: | The NeXtMidas macro/primitive to receive the X-Midas message must have been started from an X-Midas macro with "`NM <command>/ID=<id>`".
Import the X-Midas option tree at the start of the JVM `(nm/on/bg/id=999)`. Learn the position of graphical NeXtMidas primitives relative to X-Midas graphic windows.
All NeXtMidas commands default to using NeXtMidas pipes. This is because the NeXtMidas pipes provide better performance and more flexibility than X-Midas pipes. If you want to use an X-Midas pipe from NeXtMidas you need to prefix the X-Midas pipe name with `xmpipe:` (e.g. use `xmpipe:_mypipe` to read the X-Midas pipe named `_mypipe`).
1. #### Building Building NeXtMidas as an X-Midas option tree is simple: Define `NMROOT`, the root install directory of NeXtMidas. On Unix for example,
       $ setenv NMROOT /home/smith/nxm210
2. Add the NeXtMidas SYS directory as an X-Midas option tree named NXM
       X-Midas> xmopt nxm $NMROOT/nxm/sys
3. Add NXM to your path
       X-Midas> xmp +nxm
4. Build the UCL option
       X-Midas> xmbopt ucl
5. Next, build the NXM option
       X-Midas> xmbopt nxm
6. Run the confidence tests.
       X-Midas> home $NMROOT/nxm/sys/test
       X-Midas> mtrans %confidence
       X-Midas> %confidence
Note: Always use `XMBOPT NXM` command to build the NXM option tree. Other build commands will not properly build and link the JNI libraries required. #### Running NeXtMidas Commands from X-Midas An example macro is:
    X-Midas> sd360/nmplot 1e9
1. Look for additional examples in `$NMROOT/nxm/sys/mcr/` or on the Using NeXtMidas as an X-Midas Option Tree Demos page. The `NM` command is used to start a Java Virtual Machine (JVM) in which all subsequent NeXtMidas commands are launched. The steps for running a NeXtMidas command are: Turn on the JVM
       X-Midas> nm/on/bg/id=999
2. Note: The /ID=[n] switch is optional but is required for subsequent commands to send and receive messages. The value of the ID is not important. Run the NeXtMidas command(s)
       X-Midas> nm plot nxm.sys.dat.world|nxm.sys.dat.coneflags5000
       X-Midas> nm plot nxm.sys.dat.Sunset.jpg
       X-Midas> nm plot nxm.sys.dat.alaska view=xyz
3. Turn off the JVM
       X-Midas> nm/off
Inside an X-Midas macro this would look like:
    startmacro
      ...
      nm/on/bg/id=999  ! Start the JVM, permitting messages with /ID switch
      xpipe/setup on
        ...
        ! Run some NeXtMidas commands
        nm plot nxm.sys.dat.world
        ...
      xpipe off
      nm/off    ! Stop the JVM
      ...
    endmacro
#### Using X-Midas Results from NeXtMidas The global table `XM` in NeXtMidas maps directly to the X-Midas results table.
    X-Midas
    X-Midas> res temp 123.0
    NeXtMidas
    nM> res/global xm.temp
    D: XM.TEMP      = 123.0
#### Using X-Midas Pipes from NeXtMidas NeXtMidas can directly use X-Midas pipes. To prevent confusion all X-Midas pipes are given an `xmpipe:` prefix in NeXtMidas. For example:
            startmacro
              nm/on/bg/id=999  ! Start the JVM, permitting messages with /ID switch
              xpipe/setup on
                waveform _mypipe sf 1e9
                nm plot xmpipe:_mypipe
              xpipe off
              nm/off    ! Stop the JVM
            endmacro
#### Using X-Midas Messages from NeXtMidas X-Midas messages are automatically mapped to NeXtMidas messages using the XBC message mapping tools (see XBC: Sending Messages to a Host Primitive for details). Many examples are included in the following files: `$NMROOT/nxm/sys/test/test_xmmessage.txt` - X-Midas Macro `$NMROOT/nxm/sys/test/test_xmmessage.mm` - NeXtMidas Macro `$NMROOT/nxm/sys/prim/testxmmsg.java` - NeXtMidas Primitive #### Using X-Midas AUX List from NeXtMidas The NeXtMidas AUX list is directly linked to the X-Midas AUX list. Any changes made in NeXtMidas show up in X-Midas and any changes made in X-Midas show up in NeXtMidas. In addition NeXtMidas maps the familiar AUX names to their respective X-Midas counterparts:
NeXtMidas | X-Midas
--- | ---
HOME | Mc.home_path
CWD | 0
HOMEPATH | 98
DAT | 99
RAM | *
XMPIPE | *
* X-Midas has no parallel to RAM or XMPIPE. These are special AUX entries that only NeXtMidas can see. Since they are required for the correct functioning of the pipe system, they will always be included on the READ AUX in NeXtMidas.
Note that AUX entries 0, 98 and 99 have special handling in X-Midas. In particular, changes to AUX 0 made after NeXtMidas has been started (e.g. via a `cd` in the shell) will not be visible to NeXtMidas. #### Concept and Design The design is loosely modelled after the X-Midas `RSM` concept. That is to turn on the JVM, run the Java commands, and then turn off the JVM. The `NM` command uses Inter Process Communication (IPC) to signal events between X-Midas and the shared JVM. Turning on the JVM - This starts a Java Virtual Machine (JVM) and connects to the X-Midas global sections for access to the X-Midas results table, message queue and pipes. Running a Command - The JVM is capable of interpreting NeXtMidas commands and running them in the same (single) JVM via IPC. The NeXtMidas command in run in its own Java thread inside the JVM. It has access to the X-Midas resources. Turning off the JVM - This signals (via IPC) the JVM to clean up and shut down The `NM;NXM` command has the following switches:
            Global Switches
            /BG     - Runs the command in the background with respect
                      to the X-Midas shell.
            /PN=str - Sets the process name.
            /XS=n   - Allows a graphical NeXtMidas primitive to LEARN
                      window positions (see XPIPE).

            Standard Switches
            /ON     - Turns the shared JVM on (must be used with /BG).
            /OFF    - Turns the shared JVM off.
            /STATUS - Reports the status of the shared JVM.

            Debugging Switches
            /DEBUG  - Turns on general debugging messages.
            /JDEBUG - Turns on debugging messages in the JNI code.

            Special-Purpose Switches
            /CP=str       - Adds elements on the front of the classpath.
            /JARS=str     - Adds JAR files to the end of the classpath.
            /VERBOSEJNI   - Sets the "-verbose:jni" flag for the Java VM.
            /VERBOSEGC    - Sets the "-verbose:gc"  flag for the Java VM.
            /VERIFYALL    - Sets the "-Xverify:all" flag for the Java VM.
            /XCHECKJNI    - Sets the "-Xcheck:jni"  flag for the Java VM.
            /JVMOPTIONS=s - Sets special Java VM flags.
- #### Debugging / Troubleshooting Usually debugging Java applications it much easier than debugging those written in C, C++ or Fortran. This is because Java has a well-developed exception handling model that allows the application to detect and (frequently) recover from most errors. When running as an X-Midas option tree NeXtMidas needs to go outside of the standard Java safety-net in order to interact with X-Midas. This makes is much easier for errors to occur that will result in core dumps or the application hanging. While every effort has been made to aviod this situation, developers are likely to see it from time to time. Below is the list of the things most-likely to cause problems: NeXtMidas primitive causes an exception in the `open()` method. -- Causes application to hang while waiting at the X-Midas `M$SYNC`. NeXtMidas primitive uses an X-Midas pipe incorrectly (see list of limitations above). -- Causes a Signal 11 (core dump) inside X-Midas. IPC is not responding or out of memory (either due to previous core dumps or other processes using IPC incorrectly). -- Causes application to die following a `WARNING: JNI.C: Got back a strange answer via IPC` message or an IPC error message saying it could not create a message queue because it is out of memory. To help identify where things are going wrong, add the `/JDEBUG/DEBUG` switches to the `NM` command. This will turn on normal debugging (`/DEBUG`) plus debugging for the JNI code (`/JDEBUG`). Most of the time this will help determine the point where the problem first occurs. What to do to recover from a problem depends on the type of problem. Here are some possible solutions to common problems: Problem: Can't Run NeXtMidas Commands Always try running the following command first:
      X-Midas> NM/OFF
- Problem: Application Hangs If running in an `XPIPE` try using `END MACRO` first. If you are able to use `END MACRO`, run `NM/OFF` at the X-Midas prompt (this will usually clean up everything). If not in an `XPIPE` or `END MACRO` does not work, use `CONTROL+C` to kill the current process. After a `CONTROL+C` kill all processes named `nm.exe` and run:
      $ $NMROOT/os/unix/ipcclean
- Problem: Application Core Dumps Exit X-Midas with `XMEND`, kill all processes named `nm.exe` and run:
      $ $NMROOT/os/unix/ipcclean
- Problem: IPC is Not Responding or IPC is Out of Memory Try this first: Exit X-Midas with `XMEND`, kill all processes named `nm.exe` and run:
      $ $NMROOT/os/unix/ipcclean
  If that doesn't work: Stop all X-Midas sessions with `XMEND`, kill all stale X-Midas processes, close other applications you are using (i.e. any started with your USER ID) using IPC and run:
      $ $NMROOT/os/unix/ipcclean
- Problem: Getting "WARNING: jni$startjsm: No space left on device" message when running `NM/ON/BG/ID=999` You have hit the max number of message queue from Linux (default of 16). Run `ipcs -l` to see your current OS limit for number of message queues. e.g.
      $ ipcs -l
      ...
      ------ Messages: Limits --------
      max queues system wide = 16                 // kernel.msgmni
      max size of message (bytes) = 65536         // kernel.msgmnb
      default max size of queue (bytes) = 65536   // kernel.msgmax
  Edit /etc/sysctl.conf (requires root/sudo access) and change/add the `kernel.msgmni` property, e.g.
      # increase the number of message queues (from default of 16)
      kernel.msgmni = 1024
  Run the following to re-load the updated /etc/sysctl.conf (requires root/sudo access):
      $ sysctl -p
- Problem: Getting a "too large for message queue" ERROR and X-Midas macro goes into suspend mode. You have hit the max default message queue size allocated for your X-Midas macro (normally 512 slots of 8-bytes = 4096 bytes). You can increase the default message queue size using the `/NMQ=` switch on your `startmacro` line or your `message up` line in your macro. Since NeXtMidas 2.7.1, messages sent from NeXtMidas Primitives are automatically translated and normally takes up 540 slots (4320 bytes) and forward to the interested X-Midas macro/command. It is very common for X-Midas macros that needs to send and receive messages from NeXtMidss to use a larger X-Midas message queue size. For example the $NMROOT/nxm/sys/mcr/sd360.txt macro sets message queue to 8K slots (or 65536 bytes)
      startmacro/nmq=8k/ps=128k
  -- or on the `message up` line in your .txt macro:
      message/nmq=8k up
#### Top Ten Things To Remember These are the top ten tips to remember when running NeXtMidas commands from X-Midas. X-Midas will not let you use AUX paths or NMROOT with upper-case letters, URLs, or length greater than 80 characters. NeXtMidas pipes are the default in NeXtMidas, use the `xmpipe:` prefix to use an X-Midas pipe in NeXtMidas. Also make sure that `/NP=` is large enough to account for the pipes created by NeXtMidas. NeXtMidas messages are the default in NeXtMidas, when sending a message to X-Midas use `ID=XM.<XmId>` and remember to include `ARGS=` and `TYPE=` in the table for unconfigured messages. X-Midas only allows numbered AUXs. X-Midas does not allow NeXtMidas commands run from an X-Midas macro to have a named ID. X-Midas messages are painful to use. NeXtMidas trys to make them earier to use but it can't do magic (at least not in this release). When run from X-Midas the normal NeXtMidas startup files (including `nmstartup.mm`) are bypassed. X-Midas option trees and xmpath are automatically imported into NeXtMidas Use X-Midas /pname=<desired_process_name> switch to name your the NM;NXM process, e.g. `NM/ON/BG/ID=999/PNAME=NXM_JVM` to name the shared JVM for NeXtOpt "NXM_JVM". Don't forgot to call `NM/OFF` to shutdown NeXtMidas and the shared JVM.