Cloud Cruiser became HPE Consumption Analytics on Nov. 1st. You'll still see the old name in places while we update this site.

 

 

Consumption Analytics Documentation

Home > Cloud Cruiser 4 > Collecting, transforming, and publishing > ETL workbooks > Working with flows > Transformation processors

Transformation processors

Table of contents
No headers

This topic describes the processors you can use in the Transformation steps of your ETL process. For more information about the fields used when defining a processor, click the question marks in the Edit Processors dialog box in HPE Consumption Analytics Portal.

If you are moving from an earlier version of HPE Consumption Analytics Portal to version 4, refer to New names for batch XML objects in the Release Notes to see how batch XML jobs in prior releases map to workbook functions in version 4. 

Processor Description
Adjust End Time

Subtracts one second from the end time. This adjusts usage times from exclusive to inclusive, which is how HPE Consumption Analytics Portal treats them when calculating charges. If your data source uses exclusive end times, use this processor to avoid overcharging.

For example, your data source provides two records for the same VM on the same day, one with an end time of 13:00:00 and another with a start time of 13:00:00, considering them to be contiguous. HPE Consumption Analytics Portal considers them to be overlapping because the second of time at 13:00:00 is included in both, and that second is charged twice. With this processor, the end time of the first record would change to 12:59:59, removing the overlap.

HPE recommends placing this processor early in your flow so that end times are fixed before any other processing that deals with usage times.

Append Dimension to Measure Name

Adds a suffix to measure names. The suffix is constructed from one or more dimension values and constants. Because dimension values differ across rows, this usually results in rows with different measure names.

You can specify a single target measure or, with advanced options, use a regular expression to specify the measures to include or exclude as targets. To use more than one dimension value in the suffix, show advanced options and enter multiple formulas.

An exception record is created if the row does not contain values for all specified input dimensions.

Calculate Measure

Sets the value of a measure to the result of a mathematical expression. Common uses of this processor are to modify the rate or cost of a measure. Note that when including a measure in the Mathematical Expression field, use <measureName>_RATE to represent the measure's rate, and <measureName>_COST to represent the measure's cost. 

For example, if you want to modify the ResourceQtyConsumed measure to include sales tax of 6.25%, in the Target Measure field select ResourceQtyConsumed, and in the Mathematical Expression field enter ResourceQtyConsumed_RATE * 1.0625

Calculate Measure Cost Sets the passthrough cost (charge) of a measure to the result of a mathematical expression.
Calculate Measure Rate Sets the passthrough rate of a measure to the result of a mathematical expression.
Convert to End Event Leaves the row with an end time but no start time, making it an end event. If both start and end times exist, the start time is deleted. If only a start time exists, it is changed to be the end time.
Convert to Start Event Leaves the row with a start time but no end time, making it a start event. If both start and end times exist, the end time is deleted. If only an end time exists, it is changed to be the start time.
Create Exception Creates an exception for every row processed. Use this with a condition so that it processes only specific rows. If the Exception Limit for the transformation step containing this processor is zero, then the processor instead creates a warning and leaves processed rows in the flow.
Delete Dimensions Deletes the specified dimensions. With advanced options, you can use a regular expression to specify the names of the dimensions to delete.
Delete Measures Deletes the specified measures. With advanced options, you can use a regular expression to specify the names of the measures to delete.
Delete Measures Elements Deletes the specified measures passthrough values. With advanced options, you can use a regular expression to specify the names of the measures passthrough values to delete.
Delete Row Deletes every row processed. Use this processor with a condition so that it processes only specific rows.
Delete Unlisted Dimensions Deletes all dimensions not specified. This is the opposite of Delete Dimensions.
Delete Unlisted Measures Deletes all measures not specified. This is the opposite of Delete Measures.
Delete Zero Measures Deletes every measure with a value of 0.
Divert Row to Dataset

Copies every row processed from the current flow to the specified data set. Optionally, you can choose to remove it from the current flow, making this a move rather than a copy. This allows you to perform different processing on those rows by importing the dataset to another flow.

For example, a collection might include some rows that represent VMs and other rows that represent storage. Using a condition to filter processing to only the storage rows, you would divert them to a dataset and in another sheet, use an Import Dataset step to import them for processing.

Choose a dataset name that does not conflict with any worksheet names. When specifying input for an Import Dataset step, you select from a single list that includes both datasets and worksheets and does not distinguish between them. Duplicate names can confuse you and might also cause processing errors.

Normalize Series Names

Strips the numeric counter from the names of measures in a series and places them in separate records so that you have multiple instances of a single normalized measure. For example, when a VM has a few disks, you might have the dimensions VolumeName_1, VolumeName_2, VolumeName_3 and the measures VolumeSize_1, VolumeSize_2, VolumeSize_3. This processor would give you three records each with a VolumeName dimension and VolumeSize measure.

You use a regular expression to specify which names to process and which part of them is the counter to remove.

If the processor encounters a conflict when renaming members of the series, it creates a new row containing all measures and dimensions from the original row that are not members and the renamed measures and dimensions.

Prepend Dimension to Measure Name

Adds a prefix to measure names. The prefix is constructed from one or more dimension values and constants. Because dimension values differ across rows, this usually results in rows with different measure names.

You can specify a single target measure or, with advanced options, use a regular expression to specify the measures to include or exclude as targets. To use more than one dimension value in the prefix, show advanced options and enter multiple formulas.

An exception record is created if the row does not contain values for all specified input dimensions.

Rename Dimension Renames a dimension to a constant value. With advanced options, you can rename multiple dimensions and specify whether to create an exception when a source dimension doesn't exist.
Rename Matching Measures

Sometimes each row in a flow will have a different name for the same logical measure, such as networkEMEA, networkAPAC, and networkAmericas. Though the names are different, they contain some pattern that identifies them as the same logical thing.

This processor sets the measure to the same name in every row and extracts the variable part of the original name that represents context you need, storing that in a dimension. The Target Measure name can be the fixed pattern from the original names or something else. The result is a consistent resource name and retention of the identifying information. To continue the example, you might rename those measures to netTraffic and create a dimension called region that receives the values "EMEA," "APAC," and "Americas."

You use a regular expression to specify which measures to process and which part of their names to extract as the value of the created dimension.

If the Target Dimension already exists, its value is overwritten. If the Target Measure already exists, the source measure is removed and then the Target Measure is written to a new row along with all dimensions from the current row and the Target Dimension.

Rename Measure

Renames a measure to a constant value. With advanced options, you can rename multiple measures and specify whether to create an exception when a source measure doesn't exist.

When renaming multiple measures in advanced mode, specify a list of old and new names in the Name Table field, using the format {oldName:newName, ...}. For example, to rename used to disk.used and allocated to disk.allocated you would specify {used:disk.used, allocated:disk.allocated}.

If your original field names include a dot, when using advanced mode use the format {${oldName}:${newName}, ...}. For example, to rename both vcenter.mem.sizeMB and vcenter.mem.size to vcenter.mem-res.sizeMB, you would specify {${vcenter.mem.sizeMB}:${vcenter.mem-res.sizeMB}, ${vcenter.mem.size}:${vcenter.mem-res.sizeMB}}

If your field names include a space, when using advanced mode surround your field name in quotes.

Rename Measure From Dimension Lookup Finds the value of the specified dimension in a lookup and sets the name of the specified measure to the target value from the table.
Rename Measures from Lookup Finds all measure names in a lookup and for each matching measure, changes its name to the target value in the table.
Round Measures

Rounds the value of one or more measures. With advanced options, you can use a list to specify the names of the measures to round. You can select one of the following Modes:

  • Up: Round away from zero.
  • Down: Round towards zero.
  • Ceiling: Round towards positive infinity.
  • Floor: Round towards negative infinity.
  • Half Up: Round towards nearest neighbor unless both neighbors are equidistant, in which case round up.
  • Half Down: Round towards the nearest neighbor unless both neighbors are equidistant, in which case round down.
  • Half Even: Round towards the nearest neighbor unless both neighbors are equidistant, in which case, round towards the even neighbor.
Set Dimension Sets the value of the specified dimension to a constant.
Set Dimension from Dimensions

Sets the value of a new or existing dimension. The value is constructed from one or more dimension values and constants.

An exception record is created if the row does not contain values for all specified input dimensions.

Set Dimension from Lookup

Finds the value of a source dimension in a lookup and sets the value of a new or existing target dimension to the target value in the lookup.

If exception processing is active, then any record that does not result in a match is an exception.

Set Dimension From Measure Creates one or more new dimensions from a measure.
Set End Time from Dimension Sets the end time to the value of the specified dimension. You can specify the time zone and format of the dimension value and opt to delete the dimension after setting the end time.
Set Measure Sets the value of the specified measure to a constant.
Set Measure from Dimension Sets the value of the specified measure to a number captured from the value of the specified dimension. You use a regular expression to capture the number.
Set Measure Name from Dimension

Creates a new measure, setting its name to the value of the specified dimension and setting its value to 1. If a measure with that name already exists, its value is reset to 1. With advanced options, you can choose to delete the Source Dimension afterward and to create an exception when no value exists to use for the resource name.

Depending on the number of unique values for the Source Dimension, this processor might create many new measures, greatly increasing the width of your worksheet.

Set Start Time from Dimension Sets the start time to the value of the specified dimension. You can specify the time zone and format of the dimension value and opt to delete the dimension after setting the start time.
Split Dimension

Creates new sequentially-numbered dimensions by splitting the value of an existing dimension by specified delimiters.

For example, if the value of the dimension portGroupName is dv_CUSTXYZ-1010 you might split it into several custID dimensions as follows:

  • custID_1 = dv
  • custID_2 = CUSTXYZ
  • custID_3 = 1010

You can optionally restrict the number of new dimensions to create, and optionally delete the source dimension after processing.

Split JSON Dimension Parses a dimension value in JSON format and expands property values into individual dimensions.
Truncate Dimension Truncates the values of the specified dimensions to the specified length.

 

Last modified

Tags

This page has no custom tags.

Classifications

This page has no classifications.
© Copyright 2018 Hewlett Packard Enterprise Development LP