![]() |
Consumption Analytics Documentation |
The SmartJsonCollector
reader operates in one of two modes: JSON-only and Mixed. Mixed mode is useful for when plain text is present outside of the JSON data. In this case, one or more InputField
mappings can be specified to capture data embedded in plain text. This is done by defining an InputField
to describe a portion of the text with either a regular expression or a start and end position. This input field's name can then be referred to by an OutputField
. For mixed mode to work, line endings must be present in the file. Line endings are ignored in JSON-only mode.
Typically, only output fields are needed to map data since the input objects are named and well formed. The “input name” in this case is a delimited path to an object property. For example, usage.nodeName
refers to the nodeName
property of a usage
object. JSON text would be something like {"usage": {"nodeName": "abc", "compute": 1.0 }}
.
A JSON feed is any text containing data objects in JSON format. Each top-level object in the input file is mapped to an output CC Record. JSON data is mapped using one or more OutputField
beans that refer to fields in the data. A dot notation can be used when naming JSON fields since the data structure is hierarchical. For example, given input of {"usage": {"vmname": "abc", "compute": 1.0 }}
, the compute resource can be referenced as input usage.compute
.
JSON data can be mixed with plain text. As long as there are line ends in the file, plain text can be mapped into same output record as JSON data found in the same line. This is done by defining an InputField
to describe a portion of the text with either a regular expression or a start and end position. This input field's name can then be referred to by an OutputField
.
The parent output record is cloned and resources dropped before processing each child node. The hierarchical nature of the syntax is supported by the JsonPathOutputMapping
class which is recursive and supports nesting expressions and mappings. In this way, identifiers can be accumulated for the context of a node containing usage data for one or more resources.
Input can be mapped to a mutable variable instead of an output field. These variables can then be specified in place of a literal input field where necessary. To specify a variable as input, set the input name to #
followed by the variable name.
Outputs correspond to a CC Record field and typically name an attribute as the input. Input names can employ a dot notation to pull nested values. This is convenient in many cases, but limited to the first match and not intended to satisfy recursive processing needs. For example, customer.name
as an input would pull the value of a name
attribute contained in a customer
object in the current node. When using JSONPath expressions to match nodes, the input names are relative to the matching nodes.
You can map lists of objects and lists of intrinsics using * to specify retrieving every instance of a given object or intrinsic. See the example with list in this article.
For general information about JSONPath expressions, see http://goessner.net/articles/JsonPath/.
In the following example, customer data can be mapped into identifiers that are included on all output records for the document. The service
node and its children can be mapped recursively so that the vmName
identifier appears in all records having service resources.
{ "customer" : { "name" : "ABC Corp" }, "service" : { "vmName" : "9283479", "compute" : { "osType" : "Windows 2008 R2", "ramGB" : 4, "cpuCount" : 2 }, "disks" : [ { "name" : "Disk‐1", "sizeGB" : 400 }, { "name" : "Disk‐2", "sizeGB" : 2000 } ] } }
The following CSV, with dates and times omitted to save space, represents typical output from mapping the above data sample as prescribed.
,,,,3,customer,ABC Corp,vmName,9283479,osType,Windows 2008 R2,2,ramGB,4,cpuCount,2 ,,,,3,customer,ABC Corp,vmName,9283479,diskName,Disk-1,1,diskGB,400 ,,,,3,customer,ABC Corp,vmName,9283479,diskName,Disk-2,1,diskGB,2000
"interfaces": [{"address": "192.168.101.20", "type": "fixed"},{"address": "192.168.101.22", "type": "fixed"}]
This mapping captures the address of each interface.
<bean class="com.cloudcruiser.batch.collect.JsonPathOutputMapping"> <property name="jsonPathExpression" value="$.interfaces[*]" /> <property name="outputs"> <list> <bean class="com.cloudcruiser.batch.collect.OutputField"> <property name="cctype" value="IDENTIFIER" /> <property name="input" value="address" /> <property name="label" value="ip_addr" /> </bean> </list> </property> </bean>
(c) Copyright 2017-2020 Hewlett Packard Enterprise Development LP