Attention: ATM CPU and Memory are not correctly measured!
The evaluation feature offers some graphical user interface and instrumentation to measure the latency and throughput of a given query and to create some plots.
...
Before the evaluation feature can be used the features monolithicrcp.evaluation.studio feature and latency have to be added to the Odysseus product.
...
Query File: This is the query file you want to evaluate. It should deliver at least one executable, physical query plan. Remember that the query will be executed more than once, so you should design the script to be re-executable. This means for example, the query file may not try to install new sources which are already installed. Hint: you may use something like "#DROPALLSOURCES" or "#IFSRCDEF/#IFSRCNDEF" in Odysseus Script.
New (2017.02.16): There is now the possibility to add a Setup File and a Tear Down file. Similar to JUnit these scripts are called directly before and after the run (e.g. to register sources). These scripts can ran before and after each run or onyl once before and after the whole runs (Setup/Tear down every Run).
Folder for processing results:This is a the folder where the evaluation dumps the results. For each new run of an evaluation, a new subfolder (named by date and time) is created.
...
Measure Latency: If it is activated, the evaluation job adds the his pre-transformation handler to measure the latency of the query. The pre-transformation handler changes the query files (e.g. it activates the metadata latency by adding "#METADATA Latency" to the query. Furthermore, it adds the LatencyCalculationPipe and a FileSink to each root operator of the query. Therefore, if your query has different sinks, you will also have different latency results. TODO: This has changed. Currently your sources must contain valid metadata for processing!!
Create plots for latencies: If activated, the evaluation will use the merged files to create some plots, which are stored within the plot-folder (look above)
...
Measure Resources: If this is active, the evaluation job adds his pre-transformation handler to measure the CPU and memory consumption. The pre-transformation handler changes the query by adding the "#METADATA Systemload" directive to the query. To measure the systemload, the "Systemload Metadata Feature" is required.
Create plots for CPU: If activated, the evaluation will use the merged files to create a plot of the CPU consumption.
...
Plotsettings: You can choose the export type: PDF, PNG, JPEG, or GnuPlotGNUPLOT. Furthermore you may define the dimension of the plot, its height and width.
...
If you stop a query by hand (e.g. using the Queries view) - one evaluation run is canceled and the next run will be installed and started automatically.