9. SOFTWARE AND DATA HANDLING SYSTEM

9.1 Introduction

The preliminary design of the DEIMOS software and data handling system is still in progress and will not be reviewed at this November 15, 1994 DEIMOS PDR. A separate software PDR will be held when the preliminary software design is complete, and that is currently scheduled for mid-May 1995. There are several reasons for this:

First, the UCO/Lick software manpower available to work on DEIMOS has been constrained by demands of other UCO/Lick and Keck instrumentation projects, which are only now starting to wind down. As a result, the DEIMOS software design effort to date has been extremely limited, although this effort will be ramping up as current projects ramp down.

Second, several pivotal software design choices cannot yet be decided. For example, CARA has proposed that the EPICS system (a distributed instrument control system in widespread use in high energy physics and planned for use by several major telescopes, including Gemini) be used for the Keck II telescope Drive and Control System (DCS). CARA are suggesting, although not requiring, that EPICS also be used for control of Keck II instruments such as DEIMOS. To promote the use of EPICS for Keck II instruments, the Project has invited the instrument software development teams to a CARA-sponsored EPICS training session the week of December 12. The decision on whether to use EPICS for DEIMOS awaits that session.

Third, based on the experience of software development for both HIRES and LRIS, it is important to avoid ramping up the software development effort too quickly, before the rest of the instrument design has stabilized to the point where the software functional requirements can be defined, and to conserve software manpower for the later phases of the project, where it will be most needed. This is and has been a fundamental assumption on which the current software budget and schedule is based.

Accordingly, rather than present an incomplete skeletal design for the DEIMOS software and data handling system, this chapter will provide a progress report on the current design efforts, and will:

9.2 Data Flow and Data Rates

The default data system described in Chapter 4 is essentially a scaled-up of the HIRES system, but using second-generation UCSD Controllers. The following discussion assumes a 2 x 4 mosaic of three-side-buttable 2K x 4K CCDs. Each chip has 2 readout amplifiers located at opposite ends of a single 2 kilopixel serial register, which is split in the middle. During image readout, 1024 image pixels per row will be clocked into each readout amplifier. If prescan and overscan pixels are included, there will be about 1060 pixels per amplifier per row. Thus, for full-frame readout, each readout amplifier will see 4096 rows x 1060 pixels per row, or 4.34 megapixels.

For reference, the per-pixel readout time for the HIRES CCD system is 33.5 microseconds in single-amplifier readout mode. This is adequate to ensure our minimum performance spec of 120 sec to read all chips on DEIMOS. However, for the purpose of calculating the required bandwidth for the component data paths, we assume a per-pixel readout time of only 10 microseconds to allow for the possibility of reduced multiplexing and for improvements in the performance characteristics of the CCDs and/or the UCSD system, which seem probable. As noted in Chapter 4, 10 microseconds is our new readout goal.

There are 16 amplifiers per dewar. For a full-frame readout, each amplifier will produce a data stream of 4.34 megapixels, yielding 69.44 megapixels per image. Since each digitized pixel produces 2 bytes of data, the image from each dewar will contain 138.88 megabytes (MB). Given the 10 microsecond per-pixel read time, each amplifier has a data rate of 100 kilopixels sec-1, or 200 kilobytes sec-1. Since all 16 amplifiers in each dewar will be read simultaneously, the aggregate data rate per dewar is 3.2 MB sec-1, or 6.4 MB sec-1 from both dewars.

A block diagram of the data system was shown in Figure 4.1. As noted there, the data rates reflect a 20 microsecond readtime per pixel and hence should be multiplied by two to meet our more stringent current goal of 10 microseconds. The exact number of analog boards and other components will also differ, since Figure 4.1 refers to the first-generation UCSD Controller whereas our plan is to use the second-generation Controller. Regardless, the major system components will include:

With the UCSD CCD Controllers, each timing board transmits its data to the Control Room VME Crate via a pair of fiber-optic cables contained in the telescope cable wrap. The timing board/fiber cable throughput of the current UCSD system is about 3.3 MB sec-1, while the aggregate data rate from each dewar is 3.2 MB sec-1, so capacity in this link is just adequate. With the second-generation UCSD system, the corresponding throughput will be higher, about 5 MB sec-1.

The VME Crate connects to the Instrument Computer via an industry-standard interface over a fiber link. There will have to be two such links rather than the one shown in Figure 4.1 to accommodate the goal of 10 microsecond readtime per pixel.

9.3 Data Storage, I/O, and Display

The video display will be able to display only one beam at a time. However, we plan enough video memory to keep pictures from both beams in the display memory at one time. Each 8K x 8K image contains 128 MB, for a total of 256 MB of video memory required. When pictures are stored in memory, the video display will be able to switch between them in 1 second.

Manipulation of images is a concern. If images are not stored in RAM but must be fetched from disk, processing speed will be limited by disk I/O time. Assuming current SCSI speeds of 2 MB sec-1, it will take 60 seconds to read an image off disk. This would make flatfielding and other image arithmetic time consuming. We will solve this problem first by including generous amounts of RAM in the Instrument Computer (512 Mbytes is budgeted). The second strategy is to rely on faster RAID I/O technology. This technology is 10 times faster than SCSI (20 MB sec-1) and will allow full DEIMOS images to be read off disk in only 15 seconds. RAID disks are discussed in the next section.

The video display of a mosaic of 8 large CCDs is challenging. Fortunately 2Kx2K monitors are just now becoming available. A full-up scheme might utilize a mosaic of 4 of these, with 2x2 pixel binning, plus a separate monitor that would show a blowup of a subregion at full scale. Or we might use 2 side-by-side monitors to show just the upper or lower portions of each detector, or possibly just one monitor with rapid zoom and pan. The video display problem is discussed further in the next section.

9.4 Major Areas Requiring Further Research and Evaluation

A number of aspects of the DEIMOS data handling software and hardware are pushing the limits of either our experience or current technology. As such, a vital part of our initial design work is to conduct adequate research into new and emerging technologies in disk drives, networks, and image display systems. We need an accurate understanding of the costs and capabilities of these systems in order to establish reasonable performance goals. In addition, we need to gain practical experience displaying and manipulating 8K by 8K tiled images. Accordingly, we have identified several areas of research on which we hope to make significant progress by the software PDR, and for which some limited results may be available for the November review.

9.4.1 High-Capacity

Disk Drives Image storage capacity and data rates required by DEIMOS exceed those of HIRES and LRIS by an order of magnitude. Based on our current, albeit limited, knowledge of high-speed/high-capacity disk technologies (including RAID), we believe these higher requirements can be met within the amount budgeted, which is $90K. At current prices, this would buy 60 GB of RAID disk, enough to store 250 complete exposures (both beams), or 125 exposures of both raw and flattened data.

As we do not yet have any direct experience with this class of disk drive, we propose to:

a) Survey a sample of reputable vendors for current product specifications and pricing, and if possible arrange a few product demonstrations.

b) Solicit comments from other observatories (e.g., CFHT) that have purchased RAID disks.

c) Conduct a limited literature search of relevant industry trade journals for evaluations of products and surveys of where such technology is headed.

Since this technology is changing very rapidly, it doesn't make sense to do an exhaustive amount of research this early in the project. However, prior to the software PDR, we need at least to be sure that our estimates of the costs and capabilities of these drives are valid, and will invest roughly a man-week of research effort in this area. More extensive research would then be conducted in early 1997, prior to purchase of the drives.

9.4.2 High-Bandwidth Network Technologies

The high data rates required by DEIMOS will also outstrip the capabilities of the existing ethernet networks with which we are most familiar. This affects the architecture of the DEIMOS data handling system in two areas. First, it impacts the data pathway between the CCD VME crate and the Instrument Computer (see Figure 4.1). Here, we have suggested the use of FDDI fiber links. Network technology would also impact the links between the various workstations on which DEIMOS images will be displayed and reduced.

There are several competing technologies (fast ethernet, FDDI, ATM, and others) that might address our bandwidth requirements. However, this is also a rapidly changing field, and while we are at least aware of the alternatives, we lack detailed knowledge or direct experience with any of them. In order to set reasonable bounds on our system architecture within the constraints of our budget, we need to better understand these technologies. We propose a limited research initiative, similar to what we proposed for disk drive technologies. Once again, more extensive research would be conducted in late 1996 to early 1997, prior to purchase of the final systems.

9.4.3 8K x 8K Tiled Images

One of the major challenges facing the DEIMOS software and data-handling system is the efficient display and manipulation of 8Kx 8K tiled images. Currently, most workstation screens and video display monitors have resolutions of order 1K x 1K, although 2K x 2K devices are just now becoming available. While 4x4 pixel binning could be used to display a DEIMOS image on a 2K x 2K monitor, at this level of binning faint features would become difficult to discern.

Above, we described a scheme that would utilize a 2 x 2 mosaic of four 2K x 2K video display monitors to display a full 8K x 8K image using 2x2 pixel binning. A fifth monitor would be used to display an unbinned sub-region of the image, and that sub-region could be rapidly panned within the full image. While we believe that such a system could be assembled and have budgeted accordingly, it is not clear that such an arrangement is optimal, or that an astronomer would find it convenient to interact with an image so displayed. In various discussions, it has become abundantly clear that it is extremely difficult to visualize in the abstract how one would interact with a DEIMOS image displayed on a mosaic of monitors (or on other configurations of monitors). Our current thinking is that a single 2K x 2K video display might actually suffice, provided that it had a rapid capability for panning and zooming within the full 8K x 8K image, but this is by no means certain.

To better understand the nature of the task, we are planning to conduct a limited set of experiments to display simulated 8K x 8K DEIMOS images using different types of video display resolutions and configurations (mosaiced versus non-mosaiced, binned versus panned), and obtain feedback from potential DEIMOS observers as to which configurations work best. As part of these experiments, we will contact vendors of 2K x 2K video displays and attempt to arrange demonstrations and loans of equipment. Since purchase of the final system will not occur for several years, we will limit these experiments to refine our functional requirements and will conduct a more extensive evaluation of the available hardware closer to the time of purchase. Given the critical importance of this component of the system, we believe it is worth investing several weeks of effort in these initial tests prior to the software PDR.

9.4.4 Instrument Control Environments: KTL/Music and KTL/EPICS

Until recently we have been assuming that the DEIMOS instrument control hardware and software would follow the HIRES model, since it has been shown to work reasonably well. As we are already thoroughly familiar with that model, this is the least expensive course.

It should be noted that the HIRES software model consists of a number of distinct layers, and that certain layers of the model might be replaced without changing the entire model. As noted earlier, CARA has proposed the use of the EPICS system for the Keck II Telescope control software (DCS) and is urging its use for the Keck II instruments. At present, we do not know enough about EPICS to determine whether it would be of major benefit to the DEIMOS software effort. Were EPICS to be used within DEIMOS, it would replace a layer of the messaging system (KTL/Music) that is relatively close to the low-level hardware. CARA has suggested that EPICS could overcome some of the known deficiencies and limitations of the KTL/Music system, that it would produce more reliable systems, and that it would be more maintainable since EPICS is already in use at dozens of sites while KTL/Music is (and will likely always be) used at only a few. Since the KTL/Music layer that EPICS would replace is rather removed from the level of the user interfaces, the switch would be relatively transparent to observers.

We plan to investigate EPICS further later this year, and to attend the training session provided by CARA. We anticipate spending several weeks to a month on this initial evaluation prior to the software PDR. If a clear and convincing case can be made that the benefits of switching to EPICS outweigh the costs (which are non-negligible), then we will likely switch from using KTL/Music to KTL/EPICS. Otherwise, we will proceed as originally planned in following the HIRES software model, including the use of KTL/Music.

9.5 Functional Requirement Documents for the Software PDR

A key part of the software design is a precise set of functional requirements for the major system components. These need to be completed in time for the software PDR. The following rough outline is a first cut at enumerating those requirements and also serves as an overview of the total software effort (11 major areas) in the whole project. This outline needs to be fleshed out and used as the basis for the software functional requirements document. Completion of this document is a high priority, since it drives the software preliminary design.

Software and Data Handling System Functional Requirements Outline:

A. Data readout from CCD to disk

1. Data rates: per frame and per night
2. Disk, memory, and CPU requirements
3. Data storage formats a. Space and efficiency considerations
b. Format compatibility issues
c. Header information
4. Required times for tasks a. Read out an image and store to disk
b. Move an image from disk to the Instrument Computer CPU and back
c. Move an image from disk to Data Analysis Computer CPU and back
d. Divide one image by another in either the Instrument Computer or the Data Analysis Computer.
B. Image display 1. Functional requirements
2. Sample tasks a. Display latest image or previous image
b. Change lookup table
c. Pan and zoom
d. Image arithmetic
3. Proposed solutions or options
C. Quick-look image analysis 1. Full list of required functions. Sample functions include: a. Analyze a single image using standard image statistics
b. Overlay of object ID's and z's for multislit spectra
c. Special functions that operate the instrument and/or telescope in addition to taking images: 1. Focus spectrograph on grid holes, spectral lines, or stars
2. Focus telescope on stars
3. Determine CCD gain and RO noise from an exposure series
4. Multiple exposures with no readout
5. Multiple exposures with no readout but shift of image between exposures
6. Multiple exposures with no readout but shift of telescope between exposures
2. Analyze existing quick-look packages
3. Proposed solution, or options
D. TV guider software 1. Full list of required functions. Sample functions include: a. Update TV display at adjustable frame rates
b. Change threshold and contrast 1. These should be adjustable with "knobs", not just by typing in computer commands or even using a mouse/window system c. Computer adjustable look-up tables: e.g., log scale
d. Image statistics: FWHM, total counts, telescope astigmatism, coma, other 1. Some of these variables should be continuously read out and plotted on a monitor so we can see trends vs time; also telescope focus and TV reticle focus settings e. Telescope focus routine
f. TV reticle focus routine 1. Focuses TV on reticle and redetermines TV coord system g. Superimposed scales and roses: RA/Dec, Elev/Az, arcsec ruler
h. Perform a G-stack
i. Change TV filter
j. Acquire two guidestars in two offset-guiding TVs given coordinates and desired telescope coordinates
k. Move an object from offset TV to center of slit, and reverse
2. Many of these tasks require close coordination with CARA. Devise joint plan of work
E. Object acquisition and slitmask alignment 1. Top-level performance requirements
2. Proposed scheme
3. Break down scheme into sub-tasks
4. Coordinate with CARA
F. On-line data reduction 1. Full list of required functions. Sample functions include: a. Remove bias and flat field
b. Remove geometric distortion from spectra
c. Determine wavelength calibration from arc lamp, nightsky, or twilight sky
d. Apply wavelength calibration
e. Generate overlay of expected spectral features
f. Spectral extraction
g. Astrometry routines for the telescope 1. Analyze gridholes
2. Analyze star field
3. Astrometry check on standard star field
4. Measure plate scale and command telescope to adjust it
h. Aperture photometry and compare to data file for standard star field
i. Estimate spectrograph throughput from a standard star
j. Generate a flux calibration from a spectrum
k. Apply flux calibration to spectra
l. Galaxy surface brightness profile fitting package
m. Spectral analysis: 1. Emission-line spectra: FWHM, EW, flux
2. Absorption-line spectra: EW, velocity dispersion based on stored stellar templates
n. Set up night's database: 1. Bias, darks, flatfields, standard star, twilight spectra, arcs
2. Review existing data analysis systems
3. Select primary DEIMOS image analysis environment and secondary systems if any
G. Slitmask laser cutter 1. Full list of required functions. Sample functions include: a. Read input data file and cut mask: runs on cutter CPU
b. Take RA and Dec coords and generate cutter input data file
c. Take DEIMOS pixel positions and generate cutter output data file
d. Other astrometry routines?
e. Mask verification scheme?
H. Motor and sensor control software 1. List of required functions for each mechanized device and sensor
2. Inheritance from HIRES, both hardware and software models a. Galil servo motor control hardware and software
b. Air cylinder operated devices hardware and software
c. Miscellaneous sensor hardware and software 1. Temperature
2. Liquid nitrogen levels
3. Changes needed for DEIMOS (i.e., devices without counterparts in HIRES) a. Slitmask changer and alignment mechanism
b. Micro-positioning actuators (e.g., piezo stack) for flexure compensation system
I. DEIMOS instrument user interface - TBD

J. DEIMOS instrument simulator - TBD

K. DEIMOS interface to Keck II telescope drive and control system (DCS) - TBD

L. Longterm data storage and archiving - TBD

M. Required software services for general observers

N. Set priorities

9.6 Object Acquisition and Mask Alignment

As part of the drive for efficient observing, we have outlined a semi-automated scheme for object acquisition and slitmask alignment. Software to do this will be included in the software provided with the instrument.

Acquisition starts by rotating the spectrograph to a specified position angle and moving the telescope to specified coordinates. While this is occurring, the spectrograph is also fetching the desired slitmasks and installing flat mirrors (and possibly broadband filters). The fully prepared observer will have used a DEIMOS Fortran program to calculate where pre-selected guidestars should appear in the TVs. With this information, software will adjust the telescope coordinates and instrument PA to set the stars at their proper locations. This operation will be preceded by a quick look through the illuminated TV reticles to see if the TV coordinate frames need updating.

After this coarse adjustment, the goal is that stars all over the focal plane should now be centered to within 1" accuracy. The limiting factor will be the accuracy of the observer-supplied guide star coordinates. Recall that to find objects in both TVs it is necessary often to use very faint stars. Getting coordinates this good will not be trivial.

If centering is expected to be that good, the observer will then take direct images of about 10 s duration directly through both slitmasks in rapid readout mode (40 sec or less). In addition to slitlets, the masks will also contain 4-6 alignment holes about 3" square. Predetermined alignment stars selected by the observer should be visible through these holes. In a 10 s exposure there should be enough sky that the edges of the square holes will be clearly visible on the CCD image. The software will then proceed to place each guide star into the middle of its hole. This is done on one side (Side A) by adjusting the telescope pointing and the instrument position angle. The other side (Side B) will then use the fine-adjustment mask motors. If the telescope and DEIMOS move predictably, as they should, one step should suffice to make this fine adjustment. The observer will then check by taking a second direct exposure through the mask, which the computer will again analyze. If all is well, observations should be able to begin at this point.

For most moves, the time needed to acquire stars in the offset guiders coordinates will be set by the time needed to slew the telescope and rotate DEIMOS, which are comparable and of order 1 min. Zeroing the TVs and executing the coarse adjustment should take another 30 sec. By that time the spectrograph should be fully set up and ready to take the first set of direct images. Allow 1 min to take, readout, and analyze these images and another 30 sec to make the necessary corrections to the telescope coordinates, DEIMOS' PA, and the fine-adjustment motors on side B. The final check image will take another minute, plus another 20 sec to move the grating and clear filter into place for spectroscopy. The net result is a total setup time of just over four minutes. That is the goal under optimum conditions.

In many cases the observer's guide star coordinates will not be accurate enough to place the mask alignment stars in their holes on the first try. In that case it will be necessary to take the first direct image with the slitmasks open. That should add another 2 minutes or so to the cycle, for a total setup time of 6 minutes. That is the goal under typical conditions. It also obtains when only one guidestar is visible in the TVs.

9.7 Software Effort Between Now and the Software PDR

As noted in Section 9.1, the DEIMOS software design effort will be ramping up through the end of this year. Between now and the PDR, we will be directing our efforts towards those areas of research identified in Section 9.4, which we need to resolve in order to proceed with the drafting of the functional requirements document described in Section 9.5. We plan to have a completed DEIMOS software functional requirements document to present at the software PDR, upon which the preliminary design will be based.

In addition to those activities, DEIMOS software staff (primarily Tucker), will be engaged in developing interim engineering test software that will be used for the early rotation tests of the DEIMOS model. The only other software we will be developing during this phase are small programs for generating the simulated 8K x 8K DEIMOS images, and simple benchmark and diagnostic software for testing disk drives and network hardware that we are able to obtain for evaluation purposes. DEIMOS software staff will also assist in the selection and evaluation of the vendor-supplied software packages associated with the various slitmask cutter systems currently under evaluation by the electronics and mechanical staff.

9.8 Preliminary Schedule After the Software PDR

An interim schedule has been developed which identifies major software milestones and their relationship to other project milestones over the duration of the DEIMOS project (compare to master schedule in Figure 10.1). Several of these are worth noting here:

Software Preliminary Design Review May 15, 1995

CDR for mechanical, optics, and instrument electronics Nov 15, 1995

Critical Design Review for detectors and software June 15, 1996

GO TO CHAPTER 10