Help: ICE_DRIVERS_LNX

Linux ICE-DSP Driver Installation Notes



Load the ICE software tree onto the system.  The environment symbol ICEROOT
should be defined pointing to root of this tree.  

** NOTE: The environment symbol XMAREA_ICE overrides ICEROOT for X-Midas users.

The driver source files are in the $ICEROOT/drv/lnx directory. 
To install the driver, follow these steps:

** NOTE: you must be in the /bin/csh to run this **

1. Edit the $ICEROOT/drv/lnx/icepic file to set the RAM allocation 
parameters as described below.  Follow the notes in this file to
set other installation specific flags as well. 

  RAM_START is where the ICE Ramdisk starts in MBytes
  RAM_SIZED is the size of the Ramdisk RAM in MBytes
  RAM_SIZEM is the size of the Mappable RAM in Mbytes

RAMdisk memory of RAM_SIZED MBytes starts at RAM_START. 
RAMdisk memory is followed directly by the Dynamically Mappable 
memory of RAM_SIZEM Mbytes.  Insure RAM_SIZEM >= 4 to run the 
system self tests.  These 3 numbers should add up to <= the 
maximum physical ram address in the system.

See the section below on Special RAM Handling for details.

2. cd to the $ICEROOT/drv/lnx directory and type:

  ./icepic make

to make the driver and the setram utility.

3. To install the driver, type:

  ./icepic install

This drops the kernel module start and stop scripts into the 
appropriate /etc/rc.d directories for auto initialization and
cleanup with the rest of the system device driver modules.  

To load the driver without rebooting the system type:

  ./icepic start

This script can also be used to stop or restart the driver. 

The driver is now installed.  At boot time, the system will scan the PCI bus
and start the driver for each ICE-PIC card it finds.  They will show up in the
/dev filesystem as /dev/icepic0, /dev/icepic1 ...

To change the RAM allocation without reloading the driver, use the setram 
routine in this driver directory.

To exercise the cards, use the mid-level driver libraries in the ./lib area of
this software tree.  A small test program in the ./test area may be used to run
a series of tests on the card. An X-Midas macro and primitive are also 
available to run more diagnostic tests on each card.  See test/readme.txt.

X-Midas users should enter this name in the device name slot in the
hardware configuration table as shown in the examples below:  

  PIC1==ICEPIC,DEVNO=0,IOM=xxx, 
  PIC2==ICEPIC,DEVNO=1,IOM=xxx, 
  PIC3==ICEPIC,DEVNO=2,IOM=xxx, 
or
  PIC1==ICEPIC,PCI-DEV,/dev/icepic0,,,IOM=xxx, 
  PIC2==ICEPIC,PCI-DEV,/dev/icepic1,,,IOM=xxx, 
  PIC3==ICEPIC,PCI-DEV,/dev/icepic2,,,IOM=xxx, 

See HELP PIC_OPEN for hardware file setup details.
   
********************************
 Sizing the RAM areas
********************************

Applications typically need a 1 or 2 second buffer for each input/output being used 
by each card in the system.  Thus, the necessary size for the ICE-PIC DMA memory
is largely application specific.  Normally 512M, or 5 seconds at 100Mby/sec, will
suffice.  

Midas applications should use the ICE ramdisk so that allocation problems can 
be handled at the Midas file system level rather than at the driver level.  
This allows fixing allocation problems without disturbing other applications.  
Using the MAPPABLE area, to fix an allocation problem, all applications must 
be stopped and the driver restarted.  The RAMDISK also allows other primitives 
like DATALIST and PLOT to look at the RAM buffers while in use.  

Non-Midas applications will need to use the MAPPABLE area.  If you have a mix of 
Midas/NonMidas apps, both can use the MAPPABLE area.

********************************
 Special RAM Handling in Linux
********************************

Since Intel machines cannot map contiguous PCI address space into contiguous
virtual address space, it is necessary to allocate contiguous physically 
addressable RAM buffers to support high speed dma and mmap() them to virtual 
addresses for user programs to access the data.

In Linux, this can be done by reserving a piece of physical memory for the high
speed DMA by the ICE-PIC.  This physical memory can be mapped to a ramdisk for 
allocation management and convenient access, or for on-demand mappable memory 
from the ICEPIC driver, or both.  The ramdisk is not a Linux ramdisk device, 
but a Midas specific filesystem.

The simplest way to reserve large amounts of contiguous RAM is to confine the 
OS to the lower part of memory using the mem=xxM kernel parameter.  
This leaves everything above this for the DMA RAM buffers.

If you are using LILO for the boot loader, add the append= line to an entry 
in /etc/lilo.conf as shown in the example below:  

  default=icedisk

  image=/boot/vmlinuz-2.0.34-0.6
        label=icedisk
        root=/dev/hda2
        append="mem=32M"	<-- this line added
        read-only

NOTE: Don't forget to run lilo after editing lilo.conf !!!!
This can be made the default boot label by adding the default=<label> line.

If you are using GRUB for the boot loader, edit /boot/grub/menu.lst and add
the icedisk title with the mem=xxM addition as shown in the example below:  

  color white/blue black/light-gray
  default 2
  gfxmenu (hd0,1)/boot/message
  timeout 8

  title icedisk
    kernel (hd0,1)/boot/vmlinuz root=/dev/hdb2 vga=0x314 ... showopts mem=32M
    initrd (hd0,1)/boot/initrd

This can be made the default boot label by changing the default=N line, where 
N is the zero-based title number.

In the above cases, if the system has 128Mby of memory, 32Mby will be used by 
the operating system, and the rest (96Mby starting at 32M) will be reserved 
for the ICE driver.

If you have memory above 4G in your system and a 5+ series card, it is best to use 
the space above 4G.  Most systems with 8G of memory will leave a hole between 3G 
and 4G for devices.  The physical memory will be split from 0G-3G and 4G-9G.  
For instance, to use the last 2G for the ice card,

        append="mem=7186M"	<-- this line added

For SlimPics, PIC4s and earlier cards, or on systems with < 4G of memory, it 
gets complicated.  See below for a number of case examples.

On laptops, the hot-plug PCI allocator may try to assign hardware to actual
physical memory above mem=XX instead of above physical memory.  The kernel will
not put devices below 256M.  If you have more memory than this, you should add 
a system page near the top of memory to the system's list of useable memory. 
The system will not assign hardware anywhere below the last useable page.  
So a laptop with 512M may want to use:

        append="mem=254M mem=1M@510M"		<-- this line added (pre 2.6 kernels)
 or
        append="mem=254M memmap=1M@510M"	<-- this line added (2.6+ kernels)

and be able to have 256M of ram available starting at 254M.  Why not use
1M@511 ?  We found many platforms want to put other stuff just below the top 
of physical memory, in this case (512M-~128K).

Kernels 2.6 and higher only parse mem= for the mem=X form.  The mem=X@Y or mem=X$Y
must be in a memmap= argument.  Pre 2.6 kernels do NOT support memmap= arg and 2.6+
kernels will treat mem=1M@510M the same as mem=1M so there is no universal form.
For the rest of this help, we will show mem[map]= where the 'map' is for 2.6+ kernels.

In some laptop/linux/memory combinations, this does not stop the Card Services
driver from placing the card in physical memory.  A more on /proc/pci will show
where it put the card.  To stop it from doing this, we need to reserve the 
memory region for the ice card.  The syntax is:

       append="mem=254M mem[map]=256M$254M mem[map]=1M@510M"

where the amount of reserve is first, a $ sign, and the start of reserve.

By similar argument, a machine with more than 4G of memory is only allowed
to use the region below 4G for the pre 5 series ice cards.  This syntax allows 
the system to still use the memory region above 4G.  For example:

        append="mem=2048M mem[map]=4096M@4096M"	<-- this line added

This system will have 2048M of ICE ram available starting at 2048M.  
The system may have also mapped PCI devices under the 4G region as well. 

Type
	prompt> more /var/log/boot.msg 
or
	prompt> more /var/log/dmesg 
or
	prompt> more /proc/iomem

to see the kernels use of memory and make sure to stay away from the 
regions that are (reserved).  Often, the last 256M before 4G is reserved
for PCI devices.  In that case, for a system with 8G of memory,

        append="mem=1792M mem[map]=4096M@4096M"	<-- this line added

would be necessary to leave 2048M (2G) for the ice driver, and 6G-256M for the system.

Some kernels do not support the memmap= keyword, so the ice card must use the top
of memory or some will be wasted.  The 5+ series ice cards support 64bit PCI 
addressing so the ICE ram can be put above 4G.  The 4- series ice cards will not
run using memory above 4G. 

If you have memory above 4G in your system and a 5+ series card, just
use the space above 4G which is usually free of other driver clutter.  
Most systems with 8G of memory will leave a hole between 3G and 4G for devices.
The physical memory will be split from 0G-3G and 4G-9G.  
To use the last 2G for the ice card,

        append="mem=7186M"	<-- this line added

NOTE: The driver cannot handle different maps for mixed card systems at the same time.
NOTE: The driver cannot handle maps that span the 4G address boundary.  
      The ICE ramdisk and mappable area must be all above 4G or all below 4G.

********************************
 Midas ICEDISK configuration
********************************

To configure a ramdisk in X-Midas, include the following entry in the file
diskunx.cfg in the X-Midas CFG area.  Make sure to also have an entry in 
midas.cfg for a disk auxiliary that matches the entry in diskunx.cfg. 
You must create the /???/icedisk/ directory on a real disk to contain the 
Midas headers.  The detached portion holds the pointers to physical memory.
NOTE: This is NOT a Linux ramdrive, it's just physical memory that we told
Linux to leave alone.  The book-keeping is all internal to the X-Midas code.

Example diskunx.cfg:

 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
 ! Special disk handing table for UNIX systems
 ! mount_point  type  raw_device  raw_offset  raw_size
 !
 /midas/icedisk/  RAM  /dev/mem  32M  92M 
 /sam/  CFS  /dev/rspc0  0  128M 

Example midas.cfg disk aux entry:

 M_aux_list(9)       /midas/icedisk/

Including the statement:

 PIC AUX 9

in the %xmstartup macro will then allow the PIC CREATE verbs to create files
on the ramdisk for DMAs using PICDRIVER, SOURCEPIC, and SINKPIC.

To configure a ramdisk in NeXtMidas, include the following qualifiers in nmstartup.mm
where the auxiliary is being defined:

 if env.ostype eqs "UNIX"
  set U:AUX.1 "/midas/data1/+/"				! normal aux
  set U:AUX.9 "ramd:/midas/icedisk/,RAM,500M,512M"	! ramdisk aux - specific
  set U:AUX.9 "ramd:/midas/icedisk/,RAM,AUTO,"		! ramdisk aux - automatically detect from loaded driver
 elseif env.ostype eqs "DOS"
  set U:AUX.9 "ramd:\midas\icedisk\,RAM,500M,512M"
 endif

where the comma-seperated attributes are the identifier 'RAM' followed by the 
start and size of the ramdisk.