ISCSI Test Tools

Aberdeen Proving Ground neighbors share computing project

By Dr. Samuel Harley, Michael Reil, George Bartlett, Aberdeen Test Center

The United States Army Aberdeen Test Center (ATC) is the Army’s lead test center for Land Combat systems. ATC was granted a Distributed Center by the HPCMP in response to needs for data warehousing, real time processing, and On Line Analytical Processing.

Data warehousing is the process of extracting and transforming operational data into informational data and loading it into a central data store or warehouse where it is accessible via query and analysis tools by the decision makers. On Line Analytical Processing (OLAP) uses a multidimensional view of aggregate data to provide quick access to strategic information for further analysis and enables analysts to gain insight into data through fast, consistent, interactive access to a wide variety of possible views of information. While these technologies have most generally applied non-real time to business and marketing problems, the concepts they embody are adaptable (both real time and non-real time) to test and evaluation issues associated with Army weapon systems.

Distributed Center Hardware

The High Performance Computer Modernization Program Office has provided ATC with the resources for both classified and unclassified Engineering Performance Data Marts. The unclassified machine consists of 20 nodes while the classified machine is 12 nodes. Due to current workload demographics, all 32 nodes are installed in the unclassified machine. Each node is a dual Intel Pentium Xeon 2.8GHz, with 4GB RAM. The head node and slave nodes connect to a Panasas 2.2 TB RAID-5 storage device via iSCSI. Each node is interconnected via Gigabit Ethernet. The open-source object relational database, PostgreSQL is installed. The database is stored on a 400 GB RAID-5 array, and objects (test data) are stored on the Panasas disk array, which is NFS (Network Filesystem) mounted by each node in the cluster via Gigabit Ethernet. The disk arrays are large enough to keep all relevant historical data on-line. These resources are installed and managed at the ARL MSRC at Aberdeen Proving Ground, MD.

Data Flow

Data flow is from the on-site collection instrumentation to the Engineering Data Mart. Data is moved from the individual harvesters to the concentrator and then to the Data Mart using combinations of Secure Shell (ssh), AntFlow scripts, and perl scripts. While real time automated data loading is complicated by the Kerberos human-authentication requirement for network communication with any MSRC computer, the technique developed by ATC has sufficient speed and acceptable latency to service real time needs.

Bulk data ingestion is accomplished when a test session’s data is harvested by physically extracting the solid state storage media and inserting it into a Data Harvester. The process takes approximately 10 - 20 minutes for the loading of data from one source on one test item. It was estimated that 585 minutes would be required if using a high end work station from partial uploads because the time is so large.

Techniques to harvest and upload data files as they are created are under development. It is estimated that the time lapse from file transfer from the instrumentation to membership in the Engineering Performance Data Mart will be on the order of one to two minutes.

On Line Analytical Processing

Online analytical processing (OLAP) describes a class of technologies that are designed for live ad hoc data access and analysis. The most common perception of OLAP is that its use is restricted to business processes. OLAP for Test and Evaluation is more an adaptation of the concept than of the tools.

Analysis of T&E data requires feature extraction and aggregation of time series data over a number of dimensions (project, measurement type, test item, test item family, test location, test type, environmental conditions, operator, etc) While the concept of a multidimensional database coupled with easy to use OLAP tools is attractive, the dimensionality of T&E precludes the use of available OLAP tools. Instead a custom online analytical engine built on Java has been developed. Analytical processes such as histogram analysis, time outside limits, spectral content (Fast Fourier Transform (FFT) and Power Spectral Density (PSD)), shock response modeling and data visualization have been developed or are under development.

Tests at ATC often encompass hundreds of measurements. Tests of the Marine Corps Expeditionary Fighting Vehicle (EFV) included some 400 installed sensors plus some 200 system parameters. Tests of the Stryker include several hundred data bus parameters along with 30 to 60 installed sensors. The Future Combat System will challenge testing resources far beyond the levels of complexity now experienced. Real time data validation and quality assurance are absolutely required to insure that testing at proper levels is accomplished, and that re-tests that result from corrupted or incomplete data sets are eliminated.

ATC has made significant progress toward exploiting high performance computing resources in day to day test activities. Progress has been made in data harvesting, transformation, and data warehousing. An OLAP infrastructure has been put in place and a rudimentary analyst’s toolbox has been developed. A limited number of third party tools have been incorporated. Progress has been made toward implementing more automated data quality assurance, and algorithms such as limit checks, tests for signal activity, and kurtosis calculations have been incorporated.

ISCSI Test Tools