Cloud Cruiser became HPE Consumption Analytics on Nov. 1, 2018. You'll still see the old name in places while we update this site.

 

 

Consumption Analytics Documentation

Home > Documentation for older versions > Cloud Cruiser 4 for HPE GreenLake Flex Capacity > Working in Cloud Cruiser > Onboarding FC customers > Loading usage and charge data

Loading usage and charge data

To load usage data into the Cloud Cruiser database, load that data from a customer's workbook, and then schedule regular collection of that data.  Charge data is loaded when the charge job is run.  This job is scheduled daily, and can also be run on demand for a specific customer.  Note that running the charge job for a specific customer is much faster than running it for all customers.

Loading initial data from a workbook

To load data from a workbook

  1. Click Data > Workbooks.
  2. Open the workbook you want to run.
  3. In the Workbook section of the Workbook tab click Run.
    loadData1.jpg
    The Run Workbook dialog box appears.
  4. In the Select Date field, enter the date for which this run collects and processes data.
    This date is used to request data from data sources, but because data sources behave differently, the rows retrieved can have usage start and end times that fall outside of this date.
    The select date is used as the accounting date for these rows as they are processed by the workbook and published to a schema in the Cloud Cruiser database. When you run reports, this accounting date is what you specify in the report parameters and what appears in the report.
  5. Toggle the Publish Data field to Yes if you want to publish data to the database.
    When developing or debugging data transformation steps, consider setting this option to No to avoid having to repeatedly delete data loads.
  6. Click Run.
  7. After the run finishes, click View Log at the bottom left of the dialog box to open the Monitoring window, where you can view the log for any specific collection or flow.
  8. If your workbook test run did not complete successfully, or if it did not provide the data you need, edit the workbook and then run another test.
  9. When you are done editing your workbook delete any test data loads.

Scheduling regular collection of data

After testing the ability to load data, schedule regular collection.

To schedule regular collection of data

  1. Click Data > Processing > Scheduling.
  2. In the Schedules section, click New.
    The New Schedule dialog box appears.
    newSchedule1.jpg
  3. In the Job Type field, select Workbook.
  4. In the Detail section, enter the schedule’s Name and optional Description.
  5. Toggle the Enabled field to Yes so that the schedule will be active.
  6. Optionally, select a Log Level other than Default to obtain more troubleshooting information.
  7. In the Frequency section, set the Start Date to the first date that you want the schedule to run. If you want it to stop running on a certain date, set End Date to the last date the schedule should run.
  8. In the Recurrence field and the fields that appear dynamically with that selection, specify how often and at what time the schedule will run. The choices are:
    • Interval: Run the schedule after a specified number of minutes have elapsed, starting with midnight on the Start Date. Use the Minutes Between Job Runs field to specify the interval.
    • Hourly: Run the schedule every hour at the specified number of minutes after the hour. For example, if you set Minutes After the Hour to 05, the schedule runs at 12:05, 1:05, 2:05, and so on.
    • Daily: Run the schedule every day at the specified time. For example, if you set Start Time to 22:37, the schedule runs at 10:37pm.
    • Weekly: Run the schedule every week on the specified day at the specified Start Time.
    • Monthly: Run the schedule every month on the specified day at the specified Start Time.
  9. Complete the Workbook Settings and Workbook Components sections:
    1. In the Workbook field, select the customer-specific workbook for which you want to schedule collection.
    2. In the Select Date field, choose the date against which the job will run:
      • Current Day: The day that the job runs, changing with each execution of the schedule
      • Previous Day: (default) The day before the job runs, changing with each execution of the schedule
      • Current Day Minus 2 Days through Current Day Minus 14 Days: The stated number of days before the job runs, changing with each execution of the schedule
    3. In the Workbook Components section, select which collections and flows to be run.

      If you add flows to a workbook after creating a schedule for that workbook, the new flows are listed in your schedule but are not selected. You can select the new flows when you are ready to add them to the schedule.

  10. Click OK.
    The new schedule is added to the list.

You can edit the schedule by selecting it from the list and then clicking Edit. If you want to run the schedule now, select the schedule and then click Run.

Loading charge data

The charge job processes usage data published by customer jobs and generates charge data for that usage based on rate plans.  The charge job is scheduled daily to process charges for the customer jobs that run daily, and can also be run on demand for a specific customer.  Note that running the charge job for any day in a month will process usage and charges for all days in the month.

To run the charge job for a specific customer

  1. Click Data > Workbooks.
  2. Open the _RunChargeForCustomer workbook.
  3. In the Workbook section of the Workbook tab click Run.

    The Run Workbook dialog box appears.
  4. In the Select Date field, enter the date the charge job should run for.  In the customerID field, enter the 5-digit customer ID field of the customer you want the charge job run for.
  5. Click Run.  This will start the charge job.  Next to the RunScript flow, the status of the job will be shown.  The job will go through the following states:  Initializing, Running, and Completed.

 

There is also a charge job called "Default Charge job".  This job was set up to run after any usage data was loaded or deleted.  However, this job has been disabled to avoid having the charge job run unnecessarily.  This charge job should NEVER be enabled.

To enable or disable the charge job (this should always be disabled)

  1. Click Administration > Configuration > General.
  2. Select the Billing tab.
  3. Toggle the Charge Enabled field to Yes to enable charge or No to disable it.
  4. Click Save.

 

Loading data for multiple days

There might be occasions when you need to load data for multiple days. For example, when you onboard a new customer and load the initial usage, or when you need to delete and reload usage to apply a modified rate plan. To load data for multiple days, create a schedule for the customer workbook as described in the section Scheduling regular collection of data.

To load data for multiple days

  1. If reloading data, delete the existing data loads as described in the section Deleting data loads.
  2. Click Data > Processing > Scheduling.
  3. Select the customer workbook for which you will load data.
  4. Click Run.
    The Run Workbook Schedule dialog appears.
  5. In the Select Date field, select Specific Date Range.
  6. Enter the range of dates to be loaded in the Start Date and End Date fields.
  7. Run the "On Demand Charge job" for one day in each month in the date range selected.

Deleting data loads

If you want to run collection and publish data for the same day it completed successfully without errors or warnings, you must delete its previous data load prior to running it again, otherwise the run will fail with a Duplicate load detected error. You can also delete detail and summary data from older loads to recover database storage space.

To delete a data load

  1. Click Data > Processing > Monitoring.
  2. Check whether the "On Demand Charge job" is running.  If it is, wait for it to complete before continuing with the next step.  Deleting loads while the charge job is running will significantly degrade performance of both the charge job and deleting the loads.
  3. Click Data > Processing > Published Loads.
  4. Select one or more entry rows and click Delete Load(s) at the top of the page.
    A dialog box opens, asking you to confirm the deletion.
  5. In the dialog box, click Yes.
  6. Run the charge step for one day in each month that loads were deleted.

Processing usage files uploaded manually

Customer usage files are automatically downloaded from the mail server into the /CC/cc-working/usage_files directory on the application server on a daily basis.  However, occasionally these files may need to be uploaded manually.  In this case, the files may need to be prepared for use by the customer collectors, which is normally done by the _DownloadUsage job.  This section describes how to run the scripts to prepare the files.

The following types of files need to be prepared for customer job collection:

  • 3PAR 
  • DL Servers 
  • VMware

Scripts are used to prepare these files.  The scripts can either be run from Linux or from the UI. The scripts can be found in the /CC/cc-working/scripts directory:

  • preprocess3PAR.sh 

  • preprocessDLServers.sh

  • preprocessRVToolsFiles.sh

All of these scripts take 2 parameters, the name of the directory where the customer usage files are located, i.e. Fairfax (from the full path /CC/cc-working/usage_files/Fairfax), and the select date in YYYYMMDD format. As an example, you can run the following command:

/CC/cc-working/scripts/preprocess3PAR.sh Fairfax 20170427

All of these scripts convert from UTF-16 to UTF-8 and do any other preprocessing necessary for _ProcessUsage to be able to process them.

You can also run these scripts from the scheduling page. There are 3 jobs on this page:

  • _Preprocess3PAR 

  • _PreprocessDLServers

  • _PreprocessVMware

Using the jobs on the scheduling page will allow you to run the jobs for multiple select dates. Note that you will need to edit the schedule first to replace the custUsageDir parameter with the name of the customer usage files directory.  As shown in the screenshot, you would replace "CUSTOMER" with the actual customer directory name, i.e. Fairfax, HP-EMEA-UKI-GBR-00016, etc.

Last modified

Tags

This page has no custom tags.

Classifications

This page has no classifications.

 (c) Copyright 2017-2020 Hewlett Packard Enterprise Development LP