Using NeXtMidas as an X-Midas Option Tree (NeXtOpt)
NeXtMidas may be used as an X-Midas option tree. This enables an X-Midas macro
to call NeXtMidas primitives.
- Requirements
- Current Capabilities
- Building
- Running
- Using X-Midas Results from NeXtMidas
- Using X-Midas Pipes from NeXtMidas
- Using X-Midas Messages from NeXtMidas
- Using X-Midas AUX List from NeXtMidas
- Concept and Design
- Debugging
- Top Ten Things To Remember
Install libXt-devel
(RHEL/CentOS) or appropriate package that provides required development headers for X11/Intrinsic.h.
In order to build NeXtMidas as an X-Midas option tree you will need
to use three compilers:
- A Fortran compiler (for the host primitives)
- A C compiler (for the Java Native Interface, JNI)
- A Java compiler (for NeXtMidas)
The following table shows the supported compilers and X-Midas versions.
Other combinations of compliers for X-Midas 3.8.X and 4.2.X are likely
work out-of-the-box, they are red here because (as of the time of this
writing) no one has tried them yet. It is possible that a few minor
changes will be necessary before X-Midas 3.6.5 and 3.7.1 are supported.
Any help is welcome in getting this matrix filled in.
|
Fortran Compilers |
C Compilers |
Promula |
Intel (ifort) |
PGF |
GCC |
Intel (icc) |
6.5 |
9.1 |
10.1 |
5.2 |
6.0 |
8.0 |
3.2.X |
3.4.X |
4.1.X |
10.1 |
X-Midas 4.8.X |
Yes(1) |
Yes |
Yes |
Yes(1) |
Yes(1) |
Yes |
Yes |
Yes |
Yes |
Yes |
X-Midas 4.6.X |
Yes(1) |
Yes |
Yes |
Yes(1) |
Yes(1) |
Yes |
Yes |
Yes |
Yes |
Yes |
The below X-Midas version are no longer supported. |
X-Midas 4.4.X |
Yes(1) |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
X-Midas 4.2.X |
Yes |
No |
No |
Yes(1) |
Yes(1) |
No |
Yes |
Yes(1) |
No |
No |
X-Midas 3.8.3 |
Yes |
No |
No |
No |
No |
No |
Yes |
No |
No |
No |
X-Midas 3.7.1 |
No |
No |
No |
No |
No |
No |
No |
No |
No |
No |
X-Midas 3.6.5 |
No |
No |
No |
No |
No |
No |
No |
No |
No |
No |
All versions require Java 7 or later
|
(1) User tested.
|
As of this writing, NeXtMidas primitives run from the X-Midas shell can:
- Read from X-Midas pipes.
Limitations: |
- NeXtMidas supports dynamic allocation and connection of
pipes. This is not supported when using X-Midas pipes.
- The NeXtMidas primitive must open an X-Midas pipe inside
its'
open() method, otherwise the primitive
will not see the start of the pipe.
- On the NeXtMidas side the pipe name must be prefixed
with
xmpipe: .
|
- Create and write to X-Midas pipes.
Limitations: |
- NeXtMidas supports dynamic allocation and connection of
pipes. This is not supported when using X-Midas pipes.
- NeXtMidas has no control over the size of the pipes
(pipe size is set by X-Midas).
- When a NeXtMidas primitive is creating an X-Midas pipe, it
must set the file size (note that this is not
the same as pipe size) before it opens the pipe.
- The NeXtMidas primitive must open an X-Midas pipe inside
its'
open() method.
- The X-Midas
PIPE or XPIPE must
have the /NP= set to a value large enough
to account for the pipes created by NeXtMidas primitives.
- On the NeXtMidas side the pipe name must be prefixed
with
xmpipe: .
|
- Read an X-Midas pipe created by another NeXtMidas primitive. (This
will, however, be slower than just using NeXtMidas pipes.)
- Read/write results from/to the X-Midas results table.
Limitations: |
- NeXtMidas supports result groups or collections called
TABLES and can have result objects. These features are not
supported in the X-Midas results table.
|
- Read and set the AUX list to find and create files. NeXtMidas has all capabilites
X-Midas has to dynamically read and set the AUX list.
Limitations: |
- NeXtMidas supports AUX names as strings, but X-Midas
only allows numbers.
- NeXtMidas supports URLs but X-Midas only allows local
directories.
|
- Send messages from X-Midas to NeXtMidas and send messages from
NeXtMidas to X-Midas.
Limitations: |
- The NeXtMidas macro/primitive to receive the X-Midas
message must have been started from an X-Midas macro
with "
NM <command>/ID=<id> ".
|
- Import the X-Midas option tree at the start of the JVM
(nm/on/bg/id=999)
.
- Learn the position of graphical NeXtMidas primitives relative to
X-Midas graphic windows.
All NeXtMidas commands default to using NeXtMidas pipes. This is because the NeXtMidas
pipes provide better performance and more flexibility than X-Midas pipes. If you want
to use an X-Midas pipe from NeXtMidas you need to prefix the X-Midas pipe name with
xmpipe: (e.g. use xmpipe:_mypipe to read the X-Midas pipe
named _mypipe ).
|
Building NeXtMidas as an X-Midas option tree is simple:
- Define
NMROOT
, the root install directory of NeXtMidas. On Unix for
example,
$ setenv NMROOT /home/smith/nxm210
- Add the NeXtMidas SYS directory as an X-Midas option tree named NXM
X-Midas> xmopt nxm $NMROOT/nxm/sys
- Add NXM to your path
X-Midas> xmp +nxm
- Build the UCL option
X-Midas> xmbopt ucl
- Next, build the NXM option
X-Midas> xmbopt nxm
- Run the confidence tests.
X-Midas> home $NMROOT/nxm/sys/test
X-Midas> mtrans %confidence
X-Midas> %confidence
Note: Always use
XMBOPT NXM
command to build the NXM option tree. Other
build commands will not properly build and link the JNI libraries required.
An example macro is:
X-Midas> sd360/nmplot 1e9
Look for additional examples in
$NMROOT/nxm/sys/mcr/
or on the
Using NeXtMidas as an X-Midas Option Tree
Demos page.
The NM
command is used to start a Java Virtual Machine (JVM) in which all
subsequent NeXtMidas commands are launched. The steps for running a NeXtMidas command are:
- Turn on the JVM
X-Midas> nm/on/bg/id=999
Note: The /ID=[n] switch is optional but is required for subsequent commands to
send and receive messages. The value of the ID is not important.
- Run the NeXtMidas command(s)
X-Midas> nm plot nxm.sys.dat.world|nxm.sys.dat.coneflags5000
X-Midas> nm plot nxm.sys.dat.Sunset.jpg
X-Midas> nm plot nxm.sys.dat.alaska view=xyz
- Turn off the JVM
X-Midas> nm/off
Inside an X-Midas macro this would look like:
startmacro
...
nm/on/bg/id=999 ! Start the JVM, permitting messages with /ID switch
xpipe/setup on
...
! Run some NeXtMidas commands
nm plot nxm.sys.dat.world
...
xpipe off
nm/off ! Stop the JVM
...
endmacro
The global table XM
in NeXtMidas maps directly to the X-Midas
results table.
X-Midas
X-Midas> res temp 123.0
NeXtMidas
nM> res/global xm.temp
D: XM.TEMP = 123.0
NeXtMidas can directly use X-Midas pipes. To prevent confusion all X-Midas
pipes are given an xmpipe:
prefix in NeXtMidas. For example:
startmacro
nm/on/bg/id=999 ! Start the JVM, permitting messages with /ID switch
xpipe/setup on
waveform _mypipe sf 1e9
nm plot xmpipe:_mypipe
xpipe off
nm/off ! Stop the JVM
endmacro
X-Midas messages are automatically mapped to NeXtMidas messages using the
XBC message mapping tools (see XBC: Sending
Messages to a Host Primitive for details). Many examples are included
in the following files:
$NMROOT/nxm/sys/test/test_xmmessage.txt
- X-Midas Macro
$NMROOT/nxm/sys/test/test_xmmessage.mm
- NeXtMidas Macro
$NMROOT/nxm/sys/prim/testxmmsg.java
- NeXtMidas Primitive
The NeXtMidas AUX list is directly linked to the X-Midas AUX list. Any
changes made in NeXtMidas show up in X-Midas and any changes made in
X-Midas show up in NeXtMidas.
In addition NeXtMidas maps the familiar AUX names to their respective X-Midas
counterparts:
NeXtMidas | X-Midas |
HOME | Mc.home_path |
CWD | 0 |
HOMEPATH | 98 |
DAT | 99 |
RAM | * |
XMPIPE | * |
* X-Midas has no parallel to RAM or XMPIPE. These are special AUX
entries that only NeXtMidas can see. Since they are required for the correct
functioning of the pipe system, they will always be included on the READ AUX in
NeXtMidas. |
Note that AUX entries 0, 98 and 99 have special handling in X-Midas. In particular, changes
to AUX 0 made after NeXtMidas has been started (e.g. via a cd in the shell) will
not be visible to NeXtMidas.
The design is loosely modelled after the X-Midas RSM
concept. That is to turn
on the JVM, run the Java commands, and then turn off the JVM. The NM
command
uses Inter Process Communication (IPC) to signal events between X-Midas and the shared JVM.
- Turning on the JVM - This starts a Java Virtual Machine (JVM) and connects to the
X-Midas global sections for access to the X-Midas results table, message queue and
pipes.
- Running a Command - The JVM is capable of interpreting NeXtMidas commands and running
them in the same (single) JVM via IPC. The NeXtMidas command in run in its own Java
thread inside the JVM. It has access to the X-Midas resources.
- Turning off the JVM - This signals (via IPC) the JVM to clean up and shut down
The NM;NXM
command has the following switches:
Global Switches
/BG - Runs the command in the background with respect
to the X-Midas shell.
/PN=str - Sets the process name.
/XS=n - Allows a graphical NeXtMidas primitive to LEARN
window positions (see XPIPE).
Standard Switches
/ON - Turns the shared JVM on (must be used with /BG).
/OFF - Turns the shared JVM off.
/STATUS - Reports the status of the shared JVM.
Debugging Switches
/DEBUG - Turns on general debugging messages.
/JDEBUG - Turns on debugging messages in the JNI code.
Special-Purpose Switches
/CP=str - Adds elements on the front of the classpath.
/JARS=str - Adds JAR files to the end of the classpath.
/VERBOSEJNI - Sets the "-verbose:jni" flag for the Java VM.
/VERBOSEGC - Sets the "-verbose:gc" flag for the Java VM.
/VERIFYALL - Sets the "-Xverify:all" flag for the Java VM.
/XCHECKJNI - Sets the "-Xcheck:jni" flag for the Java VM.
/JVMOPTIONS=s - Sets special Java VM flags.
Usually debugging Java applications it much easier than debugging those written in
C, C++ or Fortran. This is because Java has a well-developed exception handling
model that allows the application to detect and (frequently) recover from most
errors.
When running as an X-Midas option tree NeXtMidas needs to go outside of the standard
Java safety-net in order to interact with X-Midas. This makes is much easier for
errors to occur that will result in core dumps or the application hanging. While
every effort has been made to aviod this situation, developers are likely to see
it from time to time. Below is the list of the things most-likely to cause problems:
- NeXtMidas primitive causes an exception in the
open()
method.
-- Causes application to hang while waiting at the X-Midas
M$SYNC
.
- NeXtMidas primitive uses an X-Midas pipe incorrectly (see list of
limitations above).
-- Causes a Signal 11 (core dump) inside X-Midas.
- IPC is not responding or out of memory (either due to previous core
dumps or other processes using IPC incorrectly).
-- Causes application to die following a WARNING: JNI.C: Got back a
strange answer via IPC
message or an IPC error message saying it could
not create a message queue because it is out of memory.
To help identify where things are going wrong, add the /JDEBUG/DEBUG
switches
to the NM
command. This will turn on normal debugging (/DEBUG
)
plus debugging for the JNI code (/JDEBUG
). Most of the time this will help
determine the point where the problem first occurs.
What to do to recover from a problem depends on the type of problem. Here are some possible
solutions to common problems:
- Problem: Can't Run NeXtMidas Commands
Always try running the following command first:
X-Midas> NM/OFF
- Problem: Application Hangs
If running in an XPIPE
try using END MACRO
first.
If you are able to use END MACRO
, run NM/OFF
at
the X-Midas prompt (this will usually clean up everything). If not in an
XPIPE
or END MACRO
does not work, use CONTROL+C
to kill the current process. After a CONTROL+C
kill all processes
named nm.exe
and run:
$ $NMROOT/os/unix/ipcclean
- Problem: Application Core Dumps
Exit X-Midas with XMEND
, kill all processes named
nm.exe
and run:
$ $NMROOT/os/unix/ipcclean
- Problem: IPC is Not Responding or IPC is Out of Memory
Try this first:
Exit X-Midas with XMEND
, kill all processes named
nm.exe
and run:
$ $NMROOT/os/unix/ipcclean
If that doesn't work:
Stop all X-Midas sessions with XMEND
, kill all stale
X-Midas processes, close other applications you are using (i.e. any
started with your USER ID) using IPC and run:
$ $NMROOT/os/unix/ipcclean
- Problem: Getting "WARNING: jni$startjsm: No space left on device" message when running
NM/ON/BG/ID=999
You have hit the max number of message queue from Linux (default of 16). Run ipcs -l
to
see your current OS limit for number of message queues. e.g.
$ ipcs -l
...
------ Messages: Limits --------
max queues system wide = 16 // kernel.msgmni
max size of message (bytes) = 65536 // kernel.msgmnb
default max size of queue (bytes) = 65536 // kernel.msgmax
Edit /etc/sysctl.conf (requires root/sudo access) and change/add the kernel.msgmni
property, e.g.
# increase the number of message queues (from default of 16)
kernel.msgmni = 1024
Run the following to re-load the updated /etc/sysctl.conf (requires root/sudo access):
$ sysctl -p
- Problem: Getting a "too large for message queue" ERROR and X-Midas macro goes into suspend mode.
You have hit the max default message queue size allocated for your X-Midas macro (normally 512 slots of 8-bytes = 4096 bytes).
You can increase the default message queue size using the /NMQ=
switch on your startmacro
line
or your message up
line in your macro. Since NeXtMidas 2.7.1, messages sent from NeXtMidas Primitives are automatically
translated and normally takes up 540 slots (4320 bytes) and forward to the interested X-Midas macro/command.
It is very common for X-Midas macros that needs to send and receive messages from NeXtMidss to use a larger X-Midas message queue size.
For example the $NMROOT/nxm/sys/mcr/sd360.txt macro sets message queue to 8K slots (or 65536 bytes)
startmacro/nmq=8k/ps=128k
-- or on the message up
line in your .txt macro:
message/nmq=8k up
These are the top ten tips to remember when running NeXtMidas commands
from X-Midas.
- X-Midas will not let you use AUX paths or NMROOT with upper-case
letters, URLs, or length greater than 80 characters.
- NeXtMidas pipes are the default in NeXtMidas, use the
xmpipe:
prefix to use an X-Midas pipe in NeXtMidas. Also
make sure that /NP=
is large enough to account for the
pipes created by NeXtMidas.
- NeXtMidas messages are the default in NeXtMidas, when sending a message
to X-Midas use
ID=XM.<XmId>
and remember to include
ARGS=
and TYPE=
in the table for unconfigured
messages.
- X-Midas only allows numbered AUXs.
- X-Midas does not allow NeXtMidas commands run from an X-Midas macro
to have a named ID.
- X-Midas messages are painful to use. NeXtMidas trys to make them earier
to use but it can't do magic (at least not in this release).
- When run from X-Midas the normal NeXtMidas startup files (including
nmstartup.mm
) are bypassed.
- X-Midas option trees and xmpath are automatically imported into NeXtMidas
- Use X-Midas /pname=<desired_process_name> switch to name your the NM;NXM process,
e.g.
NM/ON/BG/ID=999/PNAME=NXM_JVM
to name the shared JVM for NeXtOpt "NXM_JVM".
- Don't forgot to call
NM/OFF
to shutdown NeXtMidas and the shared JVM.