CORK and BPR Field Guide

Crib sheets

Simple health check and download

1. Make sure laptop clock is synchronized (see Sync to NTP)

2. Create a working directory (see Setup working directory)

3. Connect instrument to computer (see Connecting the Instrument)

4. Turn on communication and power after connection is established make sure power and comms are off before ODI connector is removed from parking position

5. Run mlterm, making sure to use a reasonable log-file name (see Starting mlterm)

6. Get instrument stats F

   1. Take note of clock drift (do not syncronize the clock, later on)
   2. Take note of predicted download times

7. Get basic logger settings I

   1. Take note of download baud-rate (115200 for NEPTUNE instruments)
   2. Take note of control baud-rate (115200 for NEPTUNE instruments)

8. Consider to increase download speed (see Increase download speed)

9. Download data D, making sure to use a reasonable file name

10. Get instrument stats F again

11. Deploy instrument and quit Z

    PI might suggests real-time monitoring while following next steps

12. Backup data (*.raw) to USB key (see Backup data)

13. Check data integrity

    1. run mlbin xxxx.raw

       strips off file system and generate xxxx.bin (there should be no errors)

    2. run dumpBin.py -a -n -t -s xxxx.bin > xxxx.dmp

       Does some integrity checking and creates ASCII dump

    3. run tail xxxxx.dmp to see statistics

       Discuss with PI

14. Contact PI and discuss whether to clear memory

15. Backup *.log files and sent *.log and *.raw files to PI

16. Make sure the communication and power lines are turned off before the under water mateable connector is disconnected.

Howto’s

Linux

Working with the HP-Mini s that are used to download data from CORK s and BPR s requires some basic Linux skills. This section provides some recipes for common tasks.

Linux session with terminal window.

Open a terminal window

Click on the terminal icon

to open a command-line terminal window. If needed you can create tabs with additional terminals from the File menu.

Basic Linux Commands

pwd

Prints the current working directory

ls

Lists the files in the current directory. Use ls -lh to get more details about the files.

mkdir

Creates a new subdirectory (e.g. mkdir TestDirectory)

rm

Deletes specified files. rm -r also deletes directories. Careful, there is no way back!!!

cd

Change into specified directory. As with all other commands, pressing Tab does an auto-complete–try it!

Setup working directory

Before you start to communicate with a CORK or BPR, create a working directory under /home/corks/Cruises or /home/corks/Tests–what ever seems more appropriate. If you are on a cruise you might consider creating subdirectories for individual instruments. In a terminal window change the directory to the working directory and execute cp /home/corks/corkobservatory/CalibrationFiles/\*.txt ./ to copy some calibration files into your working directory (see mlterm calibration files). Now, you’re all set to begin communication with the logger, data download, and post-processing.

If all of this is not clear to you, you might want to read Open a terminal window and Basic Linux Commands first.

Backup data

After downloading data from an instrument and before clearing the memory, make sure that you make at least one backup copy of the data is on a USB key. To make a backup follow these steps:

1. Insert USB key (it will be mounted under /media)

2. Find out the name/directory of the drive (ls /media)

3. Copy files cp file1.raw [file2.log ...] /media/nameOfUSB (cp -R let’s you copy directories recursively)

4. Unmount USB key (click computer icon, hover over device–e.g. KINGSTON, click eject symbol)

   

5. Now the eject symbol should have disappeared and you can safely remove the device

Sync to NTP

1. Boot up the HP-Mini and make sure that wireless networking is enabled (both LEDs on front are blue) if you do not have a cabled connection.

2. Make sure you are connected to a network

   1. Hover over icon on the bottom tool bar (globe, or wireless signal) and check if eth0 (cable) or eth1 (wireless are connected)

      

   2. If no connection is up, click on the globe and connect to a network (e.g. TGT if you happen to be on the Thompson)

3. Open a terminal window and monitor the NTP log file (tail --follow /var/log/ntp – Ctrl-c to quit)

   1. Look for synchronized to xxx.xxx.xxx.xxx, stratum ? that happened within the last 30 min or so.
   2. If you do not see anything, try to restart NTP (sudo /sbin/service ntp restart – you will need the root password)
   3. If after a few minutes you still do not see any signs of synchronization, you might have to Setup NTP

Setup NTP

Todo

Add section about NTP setup

Instrument setup

Connecting the Instrument

Most modern CORKs and BPR have an RS-422 serial interface. We found that Moxa UPort USB-to-RS422 converters provide the most reliable connections.

mlterm

Starting mlterm

Before running mlterm in a command-line terminal, make sure you are in a directory that is appropriately setup (see Setup working directory). If you simply run mlterm, the session will be logged (appended) to a logfile named after the current date. It is advisable to use the -l myLogfile.log command-line option, to choose a descriptive name. On startup, mlterm tries all possible baudrates to connecte to the instrument. If the baudrate is known (115200 for all NEPTUNE instruments) the -b 115200 option provides a quicker initialization. As shown in Linux session with terminal window., the instrument should wake up after a short period of time and after hitting Enter the high-level mlterm menu will be presented.

Increase download speed

Start mlterm (see Starting mlterm) and from high-level menu proceed as follows:

1. Enter low-level mode by pressing P and Enter
2. Bring up MT01 menu Ctrl-s
3. Choose option to change download baudrate 3
4. Choose appropriate download speed (most likely 8 230400)
5. Quit MT01 menu Q
6. Quit low-level interface Esc

Now, your are all set to start the download procedure.

Note, the download baudrate will automatically be set to it’s default if you deploy the instrument by quitting mlterm with Z or after a timeout period of 30 minutes.

Data post-processing

After downloading raw data from an instrument (see Simple health check and download) the data has to post-processed.

First, mlbin has to be applied (mlbin filename.raw) to turn the *.raw file into a *.bin file. The *.raw files still contain the filesystem of the flash card and recovery information in case there were errors in the data transmission. mlbin strips off the filesystem and tries to fix any errors that might have occured. The *.bin file only contains data records in binary counts.

mldat9 (mldat9 filename.bin) is the standart utility to calibrate *.bin data and write the results into a *.dat ASCII file. In order to use mldat9 you have to cd into the directory with the *.bin file and make sure there is also a current set of calibration files in the directory (see Setup working directory). While running mldat9 you will be ask whether the temperatures are measured using a platinum sensor or a thermistor. Platinum sensors are mostly used in CORKs and BPRs installed past 2009 and have two-digit serial numbers, while themistors have one digit serial numbers. The serial numbers are shown in the sites.txt calibratrion file (see mlterm calibration files) and should be noted in the respective *.par file under ~/corkobservatory/ParameterFiles. Furthermore, you are asked whether the Paroscientific gauge measures pressure and temperature or pressure only. If you do not know the answer you can lookup the gauge serial numbers that are mentionend in sites.txt in parosci.txt and check whether U0, Y1, Y2 are zero (pressure only) or non zeros (pressure and temperature). Then you have to select a data format (typically `` 4. Parosci temperature compensated pressure``) and a site description which is a line from the site.txt calibration file. The site description has the following format:

number:	description_rtcID	deploymentMonth	temperatureSerial	parosciSerial	parosciSerial	...
3:	1027C_89	Jul10	98	120238	120239	...

mldat9 automatically removes spikes from the paros pressure and temperature channels that get sometimes generated by the measurement system and is convinient to use, but it has certain limitations:

1. mldat9 has problems with partial records that can sometimes occur especially on systems that occasionally switch between battery and cabled mode.
2. mldat9 cannot cope with instruments that have a mix between gauges that measure temperature and pressure and pressure only. These configurations exist especially for CORKs which have many gauges and are designed to operated in cabled mode.

To cope with these problems two Python programs were developed that can be used instead of mldat9 (see Install dependencies for Python scripts). The first script dumpBin.py can be used fix *.bin files by eliminating partial records, remove spikes, find problems, and dump the binary counts into an ASCII format that is human readable and can be calibrated using a calibrateLogfile.py.

A common workflow to process filename.bin using these scripts would be:

1. Fix data problems using dumpBin.py -n -s -a -t -b filename_fixed.bin filename.bin > filename.dmp
   • filename_fixed.bin will contain all complete data records
   • filename.dmp will contain ASCII representation of binary data and debugging information
2. Dump despiked data (only works with clean data) for calibration ``dumpBin.py -d filename_fixed.bin > filename_despiked.dmp
3. Calibrate the data calibrateLogfile.py -c calibrationName filename_despiked.dmp > filename_despiked.dat
   • calibrateLogfile.py also works with data from stdin, so the last two steps can be combined using a pipe: dumpBin.py -d filename_fixed.bin | calibrateLogfile.py -c calibrationName > filename_despiked.dat
   • Run calibrateLogfile.py -l to get a list of available calibrations. New calibrations can be defined in calibrateData.py.

Run dumpBin.py --help and calibrateLogfile.py --help to study all available the commandline options.

mlterm calibration files

There are three different calibration files:

parosci.txt

Calibration coefficients for Paroscientific pressure gauges

platinum.txt

Calibration coefficients for platinum housing temperature sensors

therms.txt

Calibration coefficient housing temperature thermistors

sites.txt

Data base that associates deployment locations and times, logger IDs, and sensor IDs. This information is used for calibration routines.

Note, mldat and mlterm only support only up to 30 lines in sites.txt.

Todo

Changed mldat (wrt2dat.c – DEFINE MAXSITES) to support sites.txt files with more than 30 lines. Add same functionality to mldat (formatdata.c) requires CORK simulator for testing...

Glossary

AWQ bulkhead connector

CORK pressure cases usually have an 6 pie AWQ bulkhead connector, that connects to an 7-pin ODI or SEACON underwater mateable connector. BPR pressure cases usually have an 2 piece AWQ bulkhead connector, that connects to a 7-pin ODI connector.

ODI to two pie AWQ cable

BPR

Bottom Pressure Recorder

CORK

Circulation Obviation Retro-fit Kit

HP-Mini

There are four HP-Minis setup to do CORK and BPR downloads around the world. It is also possible to compile the communication software for Mac and there is an image for VirtualBox that makes it possible to run the software on a Linux virtual machine under windows.

Moxa UPort

The Moxa UPort USB-2-RS422 to adapter provides superior download speeds over other brands. Unfortunately, the driver only supports older versions of Linux. UPDATE: There is a new driver released in summer 2012. Also, Nic Scott mentioned wrapping Windows drivers under Linux