You are here: Home Stock Analysis Application Application Requirements
Personal tools
Document Actions

Application Requirements

by Ezio Corso last modified 2007-11-28 16:40

This is a short overview of the application as a whole.

Application Porting and Deployment

Application Porting and Deployment

1. Short Overview: General description of the purpose of the application and why the grid is being used (collaboration between partners, federation of resources, access to distributed data, etc.). (500 chars)

Transfers and unzips large quantities of data from secure storage to each node, performs identical computationally intensive statistical analysis of the data at each node and then zips and securely stores the voluminous results of this analysis. This is ideal for the primary objective of analysing a massive financial databases on an instrument-by-instrument basis (one instrument’s data analysed at each node) but may have many other application domains. This could therefore become a valuable tool for the grid community at large.

 

2. Analysis: Specific functional requirements of the application that are addressed with the proposed novel grid services or APIs. (1000 chars)

The proposed application will automatically manage the analysis of a large mass of financial data. For each financial instrument there is a zip file: its content is one text file per trading day containing high frequency time-series information for that instrument. Overall there are 4 TB of unzipped data: compression reduces it to roughly 100 GB. One analysis run consists in launching one job for each stock; for each instrument about 150 time-series are constructed and analysed; about 700 instruments will be analysed in each run. Many runs are expected, as both the analysis and the time intervals of interest will change during open-ended research on the properties of the data. For a reasonably exhaustive analysis on all the data, about 200 GB of zipped output files are expected.

 

3. Impact: Provide a short outline of the application architecture focusing on how the application interacts with grid services. (1000 chars)

The application is organised in two tiers: the first one handles the grid infrastructure, while the second one is exclusively concerned with the analysis of the data. The analysis is run in the Worker Node; it expects to have locally available a set of data files for processing, and it will produce a predefined set of local output files. The grid infrastructure code in turn consists of two parts: one to launch and monitor the analysis, and one to prepare the local environment in the WNs for the analysis. The launching and monitoring part is installed in a UI host; it accepts: a file containing the list of data to process, the analysis code to execute, and the grid output directory in a predefined secure SE. The code that prepares the WN local environment: fetches data files from the secure SE, pre-processes them, launches the analysis, clears any local temporary files, and saves them back in the SE.

 

4. Conclusions: Different information should be provided depending on whether the abstract refers to already ported applications or to planned porting of applications. (500 chars)

1. For already ported applications; what are the lessons learned from the porting and any experiences with running on the grid?. Did the grid meet your expectations?

2. For planned ports; what is the time scale to having a working application?. What is the estimated effort that will be invested?

Currently the application facilitates processes that could also be achieved by grid-scripting. This is only a starting point towards a fully fledged distributed grid-application architecture WSx-compliant integrated in the information system and ready for QoS as an application-level grid service/application for financial research. The “second tier” of the application described in (3) can be viewed as a general purpose tool that is useful to any researcher wishing to perform similar computationally intensive analyses of large volumes of data.

 

« June 2017 »
Su Mo Tu We Th Fr Sa
123
45678910
11121314151617
18192021222324
252627282930
 

Powered by Plone This site conforms to the following standards: