You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »


The DEBS 2012 Grand Challenge problem involves monitoring of large hi-tech manufacturing equipment. The overall goal of the challenge is to demonstrate the capability of the event processing system to solve the set of queries for the given data. Therefore, the problem description is divided into two major parts - the description of the available data and the description of the queries which need to be executed on the data in order to solve the problem.

Monitoring data is recorded by the manufacturing equipment itself using an embedded PC and a set of sensors. The data is recorded with 100Hz frequency. However, it is possible that the time between two consecutive data points differs significantly from 10ms. The data is available as a flat CSV file, with each line representing a single event (data point). For the convenience of the DEBS Grand Challenge participants we have provided a data generator. The data generator is a simple runnable 32-bit JAR file which consumes the CSV data file and puts its contents into events which are subsequently sent via TCP/IP to a scpecified destination address. The generator JAR file can be downloaded from this link. The generator outputs (serializes) events in Google Protocol Buffers format. We also provide source code for a very simple server capable of receiving the messages output by the generator. The schema of output events is the following:

message CDataPoint {
 required fixed64 ts     = 1; //time stamp: nanoseconds since 1st Jan 1970
 required fixed64 index  = 2; //message index
 required fixed32 mf01 = 3; //Electrical Power Main Phase 1
 required fixed32 mf02 = 4; //Electrical Power Main Phase 2
 required fixed32  mf03 = 5; //Electrical Power Main Phase 3
 required fixed32 pc13 = 6; //Anode Current Drop Detection Cell 1
 required fixed32 pc14 = 7; //Anode Current Drop Detection Cell 2
 required fixed32 pc15 = 8; //Anode Current Drop Detection Cell 3
 required uint32 pc25 = 9; //Anode Voltage Drop Detection Cell 1
 required uint32 pc26 = 10; //Anode Voltage Drop Detection Cell 2
 required uint32 pc27 = 11; //Anode Voltage Drop Detection Cell 3
 required uint32 res  = 12;
 required bool bm05  = 13; //Chem A Additive Sense
 required bool bm06  = 14; //Chem B Additive Sense
 required bool bm07  = 15; //Chem C Additive Sense
 required bool bm08  = 16; //Chem A Additive Release Valve VL26
 required bool bm09  = 17; //Chem B Additive Release Valve VL27
 required bool bm10  = 18; //Chem C Additive Release Valve VL28
 optional bool pp01  = 19;
 optional bool pp02  = 20;
 optional bool pp03  = 21;
 optional bool pp04  = 22;
 optional bool pp05  = 23;
 optional bool pp06  = 24;
 optional bool pp07  = 25;
 optional bool pp08  = 26;
 optional bool pp09  = 27;
 optional bool pp10  = 28;
 optional bool pp11  = 29;
 optional bool pp12  = 30;
 optional bool pp13  = 31;
 optional bool pp14  = 32;
 optional bool pp15  = 33;
 optional bool pp16  = 34;
 optional bool pp17  = 35;
 optional bool pp18  = 36;
 optional bool pp19  = 37;
 optional bool pp20  = 38;
 optional bool pp21  = 39;
 optional bool pp22  = 40;
 optional bool pp23  = 41;
 optional bool pp24  = 42;
 optional bool pp25  = 43;
 optional bool pp26  = 44;
 optional bool pp27  = 45;
 optional bool pp28  = 46;
 optional bool pp29  = 47;
 optional bool pp30  = 48;
 optional bool pp31  = 49;
 optional bool pp32  = 50;
 optional bool pp33  = 51;
 optional bool pp34  = 52;
 optional bool pp35  = 53;
 optional bool pp36  = 54;
 optional bool pc01  = 55;
 optional bool pc02  = 56;
 optional bool pc03  = 57;
 optional bool pc04  = 58;
 optional bool pc05  = 59;
 optional bool pc06  = 60;
 optional bool pc19  = 61;
 optional bool pc20  = 62;
 optional bool pc21  = 63;
 optional bool pc22  = 64;
 optional bool pc23  = 65;
 optional bool pc24  = 66;

The only difference between the event schema output by the generator and the event schema in the CSV file is that for the generator output the time stamp format is converted from the ISO-like representation into the UNIX-like representation - seethe above schema definition. In order to run the generator a following command has to be issued:

$ java -jar generator.jar "/path/to/data/file.dat" 1.0 localhost 8080

Where "./path/to/data/file.dat" is the path to the data file, 1.0 is the speedup factor for the data generation, localhost is the name or the IP address of the destination host, and 8080 is the port number on the destination host. One feature of the generator is that it outputs events preserving their relative occurrence time. This means that the input file is replayed at (almost) the same speed at which the original data was recorded. The replay speed can be influenced by modifying the speedup factor. Specifying a value greater than 1.0 implies a replay speed which is higher than the original data rate while specifying a values below 1.0 implies a replay speed lower than the original data rate.


Within this section we describe the queries which are part of the DEBS 2012 Grand Challenge. In our description we assume that all queries operate on the same event schema as defined above. We describe every query using a block data flow diagram with white rectangles representing operators and gray rectangles representing event streams.

Query 1

The goal of the first query is to monitor the behavior of Chem Additive sensors which themselves are responsible for the monitoring of Chem Additive Release valves - see Figure above. As all input data in this task is boolean the first operation performed by the operators 1 till 6 is to detect the change of state of each input fields (bm05 till bm10) and emit those along with time stamps of the state change occurrence. The second set of operators (7 till 9) correlates the change of state of the sensor and the change of state of the valve by calculating the time difference between the occurrence of the state changes. Whenever, the time difference increases by more than 1% within a 24hour period an alarm has to be raised. Moreover, a constant monitoring of the trend for the time difference using the least squares method for the period of 24 hours has to be performed. The trend monitoring can be either visualized or returned as a stream of plot parameters.

Query 2

The goal of the second query is to monitor the energy consumption of the manufacturing equipment. The energy consumption is recorded by the sensors mf01, mf02, and mf03. The first set of operators (operator 1 till 3) calculates the average values for each of the sensors (s1.avg-mf01 till s1.avg-mf03) as well as the relative variation (s1.rng-mf01 till s1.rng-mf03) in each of the sensors readings. Both average and variation values are calculated over the period of 1 second and are output every second.

The relative variation is used to trigger the recording of the raw values of the sensor readings -- see operator 4. Whenever the relative variation on any of the energy measuring sensors exceeds the threshold of 30%, the raw data from each of the sensors (mf01-mf03) needs to be recorded starting 20 seconds before the occurrence of the threshold violation and ending 70 seconds afterwards. If multiple violations occur with the 90 second interval, it needs to be extended so that it always captures 70 seconds of raw data after the occurrence of the last violation and 20 seconds before the occurrence of the first violation.

Finally, operators 5 till 7 record the power consumption of the manufacturing equipment within a period of one minute.

  • No labels