Versions Compared

Old Version 16

changes.mady.by.user user-ade9f

Saved on

compared with

New Version 17

changes.mady.by.user user-ade9f

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Strategies and supported fragmentation types

 

Logical operatorParallelization strategiesDescriptionSupported fragmentation typesAllows definition of endpoint
JoinAO

JoinTransformationStrategy

Uses an Hash-Fragmentation for both input streams. The fragmentation attributes are gathered from the join attributes. Note that only equals-predicates (which are concatenated with &&) are supported. The fragmented datastream is merged with an UNION Operator.

HashFragmentAO

(tick)
AggregateAO

NonGroupedAggregateTransformationStrategy

 

Uses an RoundRobin or Shuffle Fragmentation for splitting the input datastream. This strategy works with partial aggregates and merges the datastream both with an union operator and an additional aggregate operator for merging the partial aggregates. This strategy works with and without grouping. Only aggregations with one input attribute are supported.RoundRobinFragmentAO(error)

ShuffleFragmentAO

GroupedAggregateTransformationStrategy

Uses a Hash-Fragmentation for the input stream. The fragmentation attributes are gathered from the grouping attributes. So this strategy only works if the aggregate operator has an grouping. The fragmented datastream is merged with an UNION Operator.

HashFragmentAO

(tick)
Info

Bold fragmentation types shows the preferred type if nothing is defined.

...

Code Block
#PARSER PQL
#PARALLELIZATION (type=INTER_OPERATOR) (degree=4) (buffersize=AUTO) (optimization=true)
#INTEROPERATORPARALLELIZATION (id=(aggregateId:selectId) (degree=2) (buffersize=GLOBAL) (strategy=NonGroupedAggregateTransformationStrategyGroupedAggregateTransformationStrategy) (fragment=ShuffleFragmentAO)
/// avoids an semantic change of the query
#INTEROPERATORPARALLELIZATION (id=(aggregateId:selectId:true) (degree=2) (buffersize=GLOBAL) (strategy=NonGroupedAggregateTransformationStrategyGroupedAggregateTransformationStrategy) (fragment=ShuffleFragmentAO)
#RUNQUERY

windowBid = TIMEWINDOW({SIZE = [1, 'MINUTES'],
                  advance = [1, 'SECONDS']
                  }, bid)

sum_price_bidder = AGGREGATE({ID = 'aggregateId',
                              aggregations = [
                                ['SUM', 'price', 'sum_price_bidder']
                              ]                                                                   
                            GROUP_BY = ['auction'],
                              FASTGROUPING = true                                                                    
                            },
                            windowBid
                          )

selectBidder = SELECT({ID = 'selectId',PREDICATE = 'bid.bidder > 1'}, sum_price_bidder)

In this example the aggregate operator is parallelized, but it is also needed that the following selection is also parallelized. So to do this at first the id of the aggregation is set, after that, seperated with a double quote, the second id of the selection is definied as the endpoint of parallelization. There is the possibility, that the parallelization changes the semantic of the query. If you want to avoid this append :true to the pair of start and end id. By default this option is set to false. If it is enabled, a possible semantic change results in an exception.