Page tree
Skip to end of metadata
Go to start of metadata

The generic generator allows you to generate arbitrary streams from value generators based on a configuration file, called schema.txt. (TODO: Placement of this file)

This file defines:

  1. The schema of the tuple that is created by the generator
  2. For each attribute, the values.

A simple example is an increase generator (Increase) that must be configured with the start value and the increase value. The following defines an attribute a with an Increase generator starting at 0 and increasing the value time by 1. The generator will create the following values: 0,1,2,3,4,5, ... and so on

schema.txt
a Increase 0 1

To create more complex tuples, you define one attribute in a single line. A values together form the generated tuple.

Let's take a simple example configuration for a stream with an increasing timestamp starting by 0 and increasing by 1 and a value x sampled from a Weibull distribution with alpha=1 and beta=1:

schema.txt
timestamp Increase 0 1
x WeibullDistribution 1 1

For generators that generates multiple values, like the MovingCircle generator or a multivariate Gaussian distribution use just one variable, e.g., Z. The stream will contain of two attributes, e.g., Z1 and Z2.

schema.txt
Z MovingCircle 6 1.0 1.0

The generators can be access from Odysseus will the following CQL statement

ATTACH STREAM generator (timestamp DOUBLE,x DOUBLE)
WRAPPER 'GenericPush'
PROTOCOL 'SizeByteBuffer'
TRANSPORT 'TCPClient'
DATAHANDLER 'Tuple'
OPTIONS ( 'port' '54325', 'host' 'localhost', 'ByteOrder' 'Little_Endian')

or with the following PQL statement

generator := RECEIVE({source='generator',Protocol='SizeByteBuffer', Transport = 'TCPClient', DataHandler = 'Tuple', SCHEMA = [['timestamp','double'],['x','double']], options = [['port','54325'],['host','localhost'],['ByteOrder','Little_Endian']]})

Now, you can just put the given statement into Odysseus and access the continuously generated data.

Download

A download version can be found here: generator.tar.gz.

The ZIP file contains a runnable JAR file of the generator and an example schema.txt file. After download, extract the ZIP file and place the schema.txt into your home directory. To start the generator issue the following command:

java -jar generator.jar 

Parameter:

  • -p Port: The listening port
  • -s schemaFile: Location of the schema file
  • -f N: The frequency
  • -n N: Number of generated tuple
  • -o outFile: Filename to dump the generated data
  • -h: Prints the help

Available generators

See Valuegenerator for a description of the different generators. Parameters can be numbers (i.e., 1 or 1.2), arrays (i.e., 1,2,3 with a comma to separate the values) or multi arrays (i.e., 1,2,3;4,5,6 with a semicolon to mark a new row) . Optional parameters are mention in brackets.

  • Integer distribution generators:
    • BinomialDistribution: trials : int,  probability : double, [seed : long(1)]
    • EnumeratedIntegerDistribution: singletons : int[], probabilities : double[], [seed : long(1)]
    • HypergeometricDistribution: population size : int, number of successes : int, sample size : int, [seed : long(1)]
    • PascalDistribution: number : int, probability : double, [seed : long(1)]
    • PoissonDistribution: mean : double, [epsilon : double(1.0)], [max. iterations : int(10000000)], [seed : long(1)]
    • UniformIntegerDistribution: lower : int, upper : int, [seed : long(1)]
    • ZipfDistribution: number of elements : int, exponent : double, [seed : long(1)]
  • Real distribution generators:
    • BetaDistribution: alpha :  double, beta : double, [seed : long(1)]
    • CauchyDistribution: [median : double(0.0), scale : double(1.0)], [seed : long(1)]
    • ChiSquaredDistribution: degrees of freedom : double, [seed : long(1)]
    • EmpiricalDistribution: [bin count : int(1000)], [seed : long(1)]
    • EnumeratedRealDistribution: singletons : double[], probabilities : double[], [seed : long(1)]
    • ExponentialDistribution: mean : double, [seed : long(1)]
    • FDistribution: numerator degrees of freedom : double,  denominator degrees of freedom : double, [seed : long(1)]
    • GammaDistribution: shape : double,  scale : double, [seed : long(1)]
    • LevyDistribution: mean : double, scale : double,  [seed : long(1)]
    • LogNormalDistribution: [scale : double(0.0), shape : double(1.0)], [seed : long(1)]
    • MultivariateNormalDistribution: means : double[], covariances : double[][], [seed : long(1)]
    • NormalDistribution: [mean : double(0.0), standard deviation : double(1.0)], [seed : long(1)]
    • TDistribution: degrees of freedom : double, [seed : long(1)]
    • TriangularDistribution: lower limit : double, upper limit: double, mode : double, [seed : long(1)]
    • UniformRealDistribution: [lower : double(0.0), upper : double(1.0)], [seed : long(1)]
    • WeibullDistribution: alpha : double, beta : double, [seed : long(1)]
    • GaussianRandom: mean : double,  deviation : double
    • RayleighDistribution: scale : double, [seed : long(1)]
    • UniformDistribution: min : double,  max : double
  • Evolving generators:
    • Alternating: start : double, increase : double, min : double, max : double
    • Increase: start : double, increase : double
    • StepIncrease: start : double, count : double, increase : double
    • Sine: [start : double(0.0)], [increase : double(0.01)]
    • Cosine: [start : double(0.0)], [increase : double(0.01)]
    • Tangent: [start : double(0.0)], [increase : double(0.01)]
    • DirichletEtaFunction: s : double
    • Prime: start : int
  • Spatial generators:
    • MovingCircle: diameter : double, [increase : double(1.0)], [x : double(0.0), y : double(0.0)]
    • MovingPolynomial: coefficients : double[], [start : double(0.0)], [increase : double(1.0)]
    • Waypoint: waypoints : double[][], [start : int(0)]
      The Waypoint generator assumes an n-dimensional array with x/y positions, e.g., 1,2;2,3;3,4 for positions (1,2), (2,3), and (3,4)
  • Switching generators:
    • AlternatingDurationSwitch: down : double, up : double, down min duration : int, down max duration : int, up min duration : int, up max duration : int, [first switch after : int(0)]
    • Sign: [value : double(1.0)]
    • Switch: down : double, up : double, down duration : int, up duration : int, [first switch after : int(0)]
  • Value generators:
    • Time: [offset : long(0)]
    • ConstantValue: value : double
    • PredifinedValue: values : double[]
  • Random generators:
    • ISAAC: [seed : long(1)]
    • MersenneTwister: [seed : long(1)]
    • Well1024a: [seed : long(1)]
    • Well19937a: [seed : long(1)]
    • Well19937c: [seed : long(1)]
    • Well44497a: [seed : long(1)]
    • Well44497b: [seed : long(1)]
    • Well512a: [seed : long(1)]

 

 

 

 

 

  • No labels