Cloud Cruiser became HPE Consumption Analytics on Nov. 1st. You'll still see the old name in places while we update this site.

 

 

Consumption Analytics Documentation

Home > Cloud Cruiser 4 > Collecting, transforming, and publishing > JSON > Mapping data to Cloud Cruiser

Mapping data to Cloud Cruiser

The SmartJsonCollector reader operates in one of two modes: JSON-only and Mixed. Mixed mode is useful for when plain text is present outside of the JSON data. In this case, one or more InputField mappings can be specified to capture data embedded in plain text. This is done by defining an InputField to describe a portion of the text with either a regular expression or a start and end position. This input field's name can then be referred to by an OutputField. For mixed mode to work, line endings must be present in the file. Line endings are ignored in JSON-only mode.

Typically, only output fields are needed to map data since the input objects are named and well formed. The “input name” in this case is a delimited path to an object property. For example, usage.nodeName refers to the nodeName property of a usage object. JSON text would be something like {"usage": {"nodeName": "abc", "compute": 1.0 }} .

JSON feed configuration

A JSON feed is any text containing data objects in JSON format. Each top-level object in the input file is mapped to an output CC Record. JSON data is mapped using one or more OutputField beans that refer to fields in the data. A dot notation can be used when naming JSON fields since the data structure is hierarchical. For example, given input of {"usage": {"vmname": "abc", "compute": 1.0 }} , the compute resource can be referenced as input usage.compute .

Mixed format

JSON data can be mixed with plain text. As long as there are line ends in the file, plain text can be mapped into same output record as JSON data found in the same line. This is done by defining an InputField to describe a portion of the text with either a regular expression or a start and end position. This input field's name can then be referred to by an OutputField.

Recursive processing

The parent output record is cloned and resources dropped before processing each child node. The hierarchical nature of the syntax is supported by the JsonPathOutputMapping class which is recursive and supports nesting expressions and mappings. In this way, identifiers can be accumulated for the context of a node containing usage data for one or more resources.

Variables

Input can be mapped to a mutable variable instead of an output field. These variables can then be specified in place of a literal input field where necessary. To specify a variable as input, set the input name to #  followed by the variable name.

Mapping output

Outputs correspond to a CC Record field and typically name an attribute as the input. Input names can employ a dot notation to pull nested values. This is convenient in many cases, but limited to the first match and not intended to satisfy recursive processing needs. For example, customer.name as an input would pull the value of a name attribute contained in a customer object in the current node. When using JSONPath expressions to match nodes, the input names are relative to the matching nodes.

You can map lists of objects and lists of intrinsics using * to specify retrieving every instance of a given object or intrinsic. See the example with list in this article.

For general information about JSONPath expressions, see http://goessner.net/articles/JsonPath/.

Basic example

Input

In the following example, customer data can be mapped into identifiers that are included on all output records for the document. The service node and its children can be mapped recursively so that the vmName identifier appears in all records having service resources.

{
  "customer" : {
    "name" : "ABC Corp"
  },
  "service" : {
    "vmName" : "9283479",
    "compute" : {
      "osType" : "Windows 2008 R2",
      "ramGB" : 4,
      "cpuCount" : 2
    },
    "disks" : [ {
      "name" : "Disk‐1",
      "sizeGB" : 400
    },
    {
      "name" : "Disk‐2",
      "sizeGB" : 2000
    } ]
  }
}

Output

The following CSV, with dates and times omitted to save space, represents typical output from mapping the above data sample as prescribed.

,,,,3,customer,ABC Corp,vmName,9283479,osType,Windows 2008 R2,2,ramGB,4,cpuCount,2
,,,,3,customer,ABC Corp,vmName,9283479,diskName,Disk-1,1,diskGB,400
,,,,3,customer,ABC Corp,vmName,9283479,diskName,Disk-2,1,diskGB,2000

Example with list

Input

"interfaces": [{"address": "192.168.101.20", "type": "fixed"},{"address": "192.168.101.22", "type": "fixed"}]

Data mapping

This mapping captures the address of each interface.

<bean class="com.cloudcruiser.batch.collect.JsonPathOutputMapping">
  <property name="jsonPathExpression" value="$.interfaces[*]" />
  <property name="outputs">
    <list>
      <bean class="com.cloudcruiser.batch.collect.OutputField">
        <property name="cctype" value="IDENTIFIER" />
        <property name="input" value="address" />
        <property name="label" value="ip_addr" />
      </bean>
    </list>
  </property>
</bean>
Last modified

Tags

Classifications

This page has no classifications.
© Copyright 2018 Hewlett Packard Enterprise Development LP