FusionAnalytics Rev. 1.0.1

RELEASE NOTES

Status: 29-11-2011

Welcome to FusionAnalytics !

We appreciate your feedback. Please use the web form or send mail to:

These Release Notes describe what is contained in this release, provide
late-breaking news, and list additional documentation for the software.

For additional information on FusionAnalytics, please visit our website at:

https://www.fusion-analytics.com

The help for this product is now online, where it can be constantly updated
and refreshed. You can find it here:

https://intergral.atlassian.net/wiki/

For known issues and further support, please see the following web pages:

https://www.fusion-analytics.com/fa/faq.cfm
https://www.fusion-analytics.com/fa/support.cfm

———————————————–

RESOLVED ISSUES 1.0.1

FusionAnalytics Rev. 1.0.2

RELEASE NOTES

Status: 24-04-2012

Welcome to FusionAnalytics !

We appreciate your feedback. Please use the web form or send mail to:

These Release Notes describe what is contained in this release, provide
late-breaking news, and list additional documentation for the software.

For additional information on FusionAnalytics, please visit our website at:

https://www.fusion-analytics.com

The help for this product is now online, where it can be constantly updated
and refreshed. You can find it here:

https://intergral.atlassian.net/wiki/

For known issues and further support, please see the following web pages:

https://www.fusion-analytics.com/fa/faq.cfm
https://www.fusion-analytics.com/fa/support.cfm

———————————————–

RESOLVED ISSUES 1.0.2

To aid development of DataServices applications (DSAPPs), the FusionAnalytics DataServices server (FADS) File Watcher watches specific files and folders, and restarts the associated DSAPP when it detects changes.

The File Watcher watches the APML folder (named by the APML configuration element), and all subfolders of that folder. This ticket details the conditions under which FADS will restart a DSAPP.

The following are the conditions which, when met, will cause FADS to stop and restart a DSAPP.

1. Any change to APML (Application Provider Markup Language) grammar files
   • These are the XML description of a DSAPP, usually with the extension .apml, .dp, and containing Module XML root element.
2. Any change to the number of files (not just APML) in the watched folder.

Changes to Data Provider programs (usually written in Groovy) will not explicitly cause a restart, although the addition or removal of a DP program file will (due to rule 2 above).

Requirement

You want to change the location of the database to a different server/port/URL/DBMS/etc.

Background

FusionAnalytics is made up of two main server-side components – the DataCollector and DataServices. The DataCollector deals with importing and quantizing data. The DataServices deals with running reports and serving data to the UI clients.

Resolution

Currently, the two server-side elements manage their DB connection configuration independently. Therefore, there are two configuration updates to make when the DB connection details change.

• Link to documentation on configuring data sources in FADC (DataCollector):
  • https://intergral.atlassian.net/wiki/pages/viewpage.action?pageId=14320040#ApplicationConfigurationFADC-datasources

• Link to documentation on configuring data sources in FADS (DataServices):
  • https://intergral.atlassian.net/wiki/pages/viewpage.action?pageId=13828146#ApplicationConfigurationFADS-DataSources

The root (index) page of the FusionAnalytics Server does not show in Internet Explorer 64bit. Only a blank / white (empty) page is shown.

To fix this issue, the security setting in Internet Explorer (64bit) that prevents Meta Tag Refresh called “Meta Tag Refresh” should be enabled. If this setting is disabled then the refresh from the FusionAnalytics root page will be prevented!

The setting can be found under the …

• Security tab > Custom Level > Miscellaneous
  … and the Meta Tag Refresh should be listed. If this is disabled, please enable it as it will stop the meta refresh from working.

Note: Aside from being able to disable it selectively, it is automatically disabled if you set IE’s security level to ‘High’ [observed in IE9].

Creating Daily Status Reports in FusionAnalytics

INTRODUCTION

A major new feature in the release candidate is the addition of a Daily Status Report.This report contains a breakdown of the server’s performance on the previous day including detailed information about the requests, database activity, resource usage and even a breakdown of individual applications running. The report can be scheduled to be emailed every day to a number of users. Details on how to setup and configure the Daily Reports can be found in the article below.

1.0 SETUP

1.1 USERS

During the installation of FusionAnalytics, you will have created an administrator account. To create other accounts, you need to go to FusionAnalytics DataServices > Users tab > Users Summary. Here you will be able to see any existing user accounts and create new ones.

To add an email address to an existing user: Click the Edit button under the username of the account that you want to add an address to. Type an email address into the email field and click the Modify User Details button.

To create a new user: Click the Create A New User button. You must fill in the username and password fields; the first name, last name and email fields are optional. You can always go back and edit these details later.

There is the option to make the account a System User, which means they will have access to all applications (this is the same settings as the administrator account). Once you have clicked Add New User, you can add any application mappings that the user will need. An account with an administrator role can make changes to the application, but an account with a user role can only view the data in a client.

1.2 SERVER SETTINGS

During installation, you will have set the host address and port number of the server for FusionAnalytics. If needed, these settings can be changed under FusionAnalytics DataServices > System tab > Server Settings. This needs to be correct for the reports (and any other background tasks) to run.

The server host field is the server or IP address that FusionAnalytics is running on; the server port is the port that FusionAnalytics is configured to use.

1.3 EMAIL SETTINGS

You will also need to make sure that the email server settings are correct before you can receive any emails from FusionAnalytics. To do this, go to FusionAnalytics DataServices > System tab > Server Settings. These fields need to be set, as any emails that are sent to and from FusionAnalytics use these as the default. This includes the sending of all notification emails.

The mail server field needs to be set to the server or IP address of the server that sends SMTP mail messages; e.g. mail.company.com:25. By default the port number is set to 25, but you can specify a port number after the address if needed; e.g. 127.0.0.1:587. If you need to provide authentication, you need to put this in front of the address: username::25.

1.4 DAILY STATUS REPORTS

The reports are configured in FusionAnalytics DataServices > Scheduled Tasks tab > Create A New Task. You can set the name, how long and how often the task will run. The task will run indefinitely if no end date is specified. There are several options for how often you want the task to run; as these are Daily Status Reports, you probably want to have them recurring daily at a suitable time, e.g. early in the morning when there are no other tasks running.

The Provider Name must be set to DailyStatus. You can then provide particular arguments to the report if needed (see 2.2 receiving reports as emails and 2.3 running reports for a specific date).

2.0 CUSTOMISING YOUR REPORTS

2.1 APPLICATON FILTERS

By making use of the application filters in the FusionAnalytics client, you can make sure you get the most out of the information you receive in your Daily Reports. They allow you to see data specifically for particular parts of your application.

Under the Administrator tab in your FusionAnalytics client, you need to select Manage Applications. Click the Add New Application button; specify an appropriate name and set the filter either by URL or by writing your own specific parameters. The filters allow you to sort the requests from each application by protocol, username, password, hostname, port or by a particular path element.

Once you have added application filters, you must log out of the client and log in again to see the applications listed. You can add as many application filters as you need; these allow you to sort the data to only see requests from particular parts of the application.

2.2 RECEIVING REPORTS AS EMAILS

To send the report straight to your inbox, you must first have an email address set on your user account. To do this, see 1.1 users. The advantage of setting this up is that every time a report is scheduled to run, it will also email a copy to any users that are specified.

Once you have ensured that the required users have email addresses, you will need to create a scheduled task (see 1.4 daily status reports). In the provider arguments section, you will need to put users in the name field and insert your username in the value field.

To send the report to multiple users, in the value field insert the usernames separated by a comma and no spaces; e.g. admin,user1,user2. All these user accounts must have email addresses set, otherwise they will not receive the report.

2.3 RUNNING REPORTS FOR A SPECIFIC DATE

You can also set the reports to run on data from a specific day. When you are creating your scheduled task, add the following provider arguments: date (for the name field) and a UTC time format in milliseconds (for the value field). There are many free UTC calculators available on the Internet to help you calculate this value. From the date entered, the report will show the previous 24 hours.

3.0 RUNNING THE REPORTS

3.1 RUNNING DAILY STATUS REPORTS INSTANTLY

The report will run at the time you have specified, but you can also force it to run at any time by clicking Run Now in the FusionAnalytics DataServices > Scheduled Tasks tab > Scheduled Tasks Summary.

Once the report has run, you will see it in your FusionAnalytics client > Business Metrics > Reports. You can use the timeline to navigate to the reports from previous dates. To open a report, right click the row and select Open Report.

3.2 READING A DAILY STATUS REPORT

By comparing data with the previous day, the Daily Status Reports allow you to see the changes in your server from day to day. This gives you information on each section and what it all means.

Overview: As the name suggests, this section allows you to quickly see vital statistics about the requests that have run in the last 24 hours. The arrows easily show if the change is positive, negative or neutral. For example, a red arrow means a negative change, but this may point up, as an increase may indicate requests are taking longer to complete.

Each attribute also shows the amount of change as a percentage, the delta value, the actual value for the last 24 hours and the value for the previous day (for the 24 prior to the current report). Several rows also list the minimum and maximum requests that occurred between each sample.

System Load Statistics: These graphs show the system and process CPU load and memory usage. If the graphs are showing that the loads or usage is high, this can indicate that the application needs to be looked at. Underneath the graphs, there is also a percentage change from the previous day, with actual and previous day values.

Session Statistics: This gives the recorded number of session and ColdFusion IDs, including the average number of requests that run per ID.

Slowest Requests: This section allows you to easily see which requests are taking the longest to complete. Only the longest 5 requests are shown, but if viewed in a web browser, this can be expanded to show the 20 slowest requests. If the same URLs constantly give slow requests, it may need to be investigated further in the application.

Slowest Requests on Average: Similar to the above section, this allows you to see the slowest requests grouped by URL. It also shows how many times this URL was used in the request count. This section can also be expanded in a web browser to see more requests.

Application Data Overview: If you have application filters configured (see 2.1 application filters), you will see them listed in this section. This gives similar information shown in the overview and session statistics sections, but only counts data that is relevant to that part of the application.

Application JDBC Overview: Similar to the above section, it lists JDBC data that is relevant to each application filter that has been configured (see 2.1 application filters).

Applications: This section divides each configured application filter into a separate sub-section, which lists the overall slowest requests and the slowest requests on average for each one. This is the same as the previous sections on slowest requests and slowest requests on average, but with data relevant to a specific part of the application.

FusionAnalytics Rev. 1.0.1

RELEASE NOTES

Status: 22-11-2011

Welcome to FusionAnalytics !

We appreciate your feedback. Please use the web form or send mail to:

These Release Notes describe what is contained in this release, provide
late-breaking news, and list additional documentation for the software.

For additional information on FusionAnalytics, please visit our website at:

https://www.fusion-analytics.com

The help for this product is online, where it can be constantly updated
and refreshed. You can find it here:

https://intergral.atlassian.net/wiki/

For known issues and further support, please see the following web pages:

https://www.fusion-analytics.com/fa/faq.cfm
https://www.fusion-analytics.com/fa/support.cfm

About this update

This update addresses the following bugs:

• FNS-18 Truncation errors while importing data
• FNS-19 ‘Value is not set for the parameter number x’ error while importing data

along with some other minor issues, and is also packaged with an upgraded version (1.0.6) of the FusionAnalytics for FusionReactor application. Once you have installed the update you can find the latest application package under [install_directory]/Applications.

You should upgrade your applications to avoid compatibility issues.

How to apply this update to your FusionAnalytics installation

To update your FusionAnalytics installation to Rev. 1.0.1 simply follow the steps below:

1. Download the Hotfix installer matching your operating system:
   

   Name	MD5
   FusionAnalytics Hotfix installer for Windows	ef2eaa8f44a84d7fe485e53b54983af0
   FusionAnalytics Hotfix installer for Windows 64	2f9c92c9cfcc714ee6385dadef71e645

2. Start the installer and follow the steps to complete the update.
   • You must accept the terms and conditions again before you are able to apply this update.
   • You must select a valid FusionAnalytics installation directory during the update process.

Your FusionAnalytics service will be restarted during the update.

Upgrading your existing 1.0.5 applications to 1.0.6

Follow the steps below to upgrade your 1.0.5 applications to 1.0.6.

1. Install the FusionAnalytics 1.0.1 update(see below for instructions).
2. In FADC, navigate to the Application Summary and make sure the application you wish to upgrade is stopped.
3. Click the Upgrade button
   
4. Ensure your application is selected in the list, press the “Choose File” button and select the new application ([install_directory]/Applications/fusionreactor-fadc-1.0.6-DCAPP.zip)
   
5. Press Upgrade. FusionAnalytics will now make the necessary changes to your database and application configuration for the new version.
   Please be patient during the upgrade process. The time taken for this process to complete will vary depending on the size of your database!

Note: You can confirm the application update was successful by looking at the Application Summary screen in FADC and checking the Revision column:

FusionAnalytics Rev. 1.0.0

RELEASE NOTES

Status: 01-10-2011

Welcome to FusionAnalytics !

We appreciate your feedback. Please use the web form or send mail to:

These Release Notes describe what is contained in this release, provide
late-breaking news, and list additional documentation for the software.

For additional information on FusionAnalytics, please visit our website at:

https://www.fusion-analytics.com

The help for this product is now online, where it can be constantly updated
and refreshed. You can find it here:

https://intergral.atlassian.net/wiki/

For known issues and further support, please see the following web pages:

https://www.fusion-analytics.com/fa/faq.cfm
https://www.fusion-analytics.com/fa/support.cfm

———————————————–

RESOLVED ISSUES 1.0.0

Problem Description

Due to a bug in FusionAnalytics 1.0.0 the import of log data targeting a BIGINT column fails if the value parsed from the log line has a length of exactly 10 characters. In such cases an error message is written to the fusionanalytics-datacollector.log and the fusionanalytics-datacollector-errors.log log files as shown in the example below:

...
2011-11-04T17:05:39.998-0400 [FADC-importer-3] ERROR [com.intergral.fusionanalytics.dc.evt.FileMonitorEventHandler] MyApp/myDataCollector/memorysummary.log: 2011-11-04 17:05:39,998 [LINE] 500 [ERROR] "The value is not set for the parameter number 6." [INPUT] "2011-11-04 16:41:50.003 1320439310000 4 1320434053992 29 2415298528 8303607808 8589934592 8303607808 42 135520024 212979712 24313856 318767104 0"
...
2011-11-04T17:05:41.130-0400 [FADC-importer-3] ERROR [com.intergral.fusionanalytics.dc.evt.FileMonitorEventHandler] MyApp/myDataCollector/realtimestats.log: 2011-11-04 17:05:41,130 [LINE] 719 [ERROR] "The value is not set for the parameter number 18." [INPUT] "2011-11-04 16:59:58.072 1320440398072 1 1320434053992 292 0 322.23214285714283 0 0 0 0 0 0 0 0 0 0 6131432480 2172175328 0.0 0 0 1 0 0.9333177780370328 0 0 725 0.0 6683287 0 1 0 0 0 0 0 0 0 0"
...
2011-11-04T17:05:41.369-0400 [FADC-importer-2] ERROR [com.intergral.fusionanalytics.dc.evt.FileMonitorEventHandler] MyApp/myDataCollector/memory-Par-Eden-Space.log: 2011-11-04 17:05:41,369 [LINE] 718 [ERROR] "The value is not set for the parameter number 9." [INPUT] "2011-11-04 17:00:00.005 1320440400001 4 1320434053992 Par-Eden-Space HEAP 0 0 2290614272 2290614272 2290614272 0 0 1 56 1299068648 2290614272 2290614272 2290614272 -1 -1 0 100 2290614272 2290614272 2290614272 2290614272"
...

Note: FADC will write a similar error message if the format of the log line does not have the expected format as defined in the DCML file. This is expected behavior and in this case only skips the line which could not be parsed.

Problem Fix

The problem can be fixed by installing FusionAnalytics Hotfix 1.0.1. Please see FNS-20 for more information.

Truncation errors while importing data

You may find that the default column widths configured by the Analytics for FusionReactor application (e.g. fusionreactor-fadc-1.0.5-DCAPP.zip) are not long enough for the data supplied by your server. If a value is too big to fit into the belonging target column the value will be truncated before it is inserted and an error message is written to the fusionanalytics-datacollector.log and fusionanalytics-datacollector-error.log files as shown below:

...
2011-11-03T10:59:45.135-0400 [FADC-importer-3] ERROR [com.intergral.fusionanalytics.dc.evt.FileMonitorEventHandler] myApp/myDC/request.log: 2011-11-03 10:59:45,135 [LINE] 6578 [ERROR] "myApp/myDC/request.log (ID=3c8ffa2a-413f-472b-b0cc-fab9c277854f): Truncated value of column JSESSIONID from 54 to 50 characters: Original value: d4309ef9d92a84dfeddc5e364d59535b6b26,+cftoken=13194335: Truncated value: d4309ef9d92a84dfeddc5e364d59535b6b26,+cftoken=13194335" [INPUT] "2011-11-03 10:57:01.692 1320332221692 4 1320259710907 98327 COMPLETED "" jrpp-124 10.10.37.249 GET http://10.0.0.10/index.cfm 93 15 8108992 1221932 8108992 6887059 "fuseaction=lms.enterClassSection&classSectionId=48506" 302 78 "" d4309ef9d92a84dfeddc5e364d59535b6b26,+cftoken=13194335 387467 85523722 0 0 0 0 0 0 0 0 93"
...

In this case you can change the column size as follows:

1. Stop the FusionAnalytics Service/Daemon
2. Open the DCML file of the application you want to change (e.g. C:FusionAnalyticsServerdatafadcApplicationsmyApplicationmyApplication.dcml) in an editor.
3. Search all columns definitions you want to update and replace their column size with the new value.

Example: Changing the size of the JSESSIONID column from 50 to 100

<FadcConfig version="1.0" xmlns="https://www.fusion-analytics.com/datacollector">
  <DataOutput dbType="mssql">
    ...
    <Table name="request_status" engine="InnoDB">
      ...
      <Column type="varchar(50)" required="false" name="jsessionid"/>
    </Table>
    ...
    <Table name="request" engine="InnoDB">
      <Column type="varchar(50)" required="false" name="jsessionid"/>
    </Table>
    ...
    <Table name="crashprotection" engine="InnoDB">
      <Column type="varchar(50)" required="false" name="jsessionid"/>
    </Table>
    <Table name="request_status_report" engine="InnoDB">
      <Column type="varchar(50)" required="false" name="jsessionid"/>
    </Table>
    ...

becomes

<FadcConfig version="1.0" xmlns="https://www.fusion-analytics.com/datacollector">
  <DataOutput dbType="mssql">
    ...
    <Table name="request_status" engine="InnoDB">
      ...
      <Column type="varchar(100)" required="false" name="jsessionid"/>
    </Table>
    ...
    <Table name="request" engine="InnoDB">
      <Column type="varchar(100)" required="false" name="jsessionid"/>
    </Table>
    ...
    <Table name="crashprotection" engine="InnoDB">
      <Column type="varchar(100)" required="false" name="jsessionid"/>
    </Table>
    <Table name="request_status_report" engine="InnoDB">
      <Column type="varchar(100)" required="false" name="jsessionid"/>
    </Table>
    ...

4. Save the updated DCML file
5. Update the columns in the database

Following the example above, you should now execute the following SQL statements:

-- MySQL
ALTER TABLE CRASHPROTECTION MODIFY JESSESIONID VARCHAR(100); 
ALTER TABLE REQUEST MODIFY JESSESIONID VARCHAR(100); 
ALTER TABLE REQUEST_STATUS MODIFY JESESSIONID VARCHAR(100); 
ALTER TABLE REQUEST_STATUS_REPORT MODIFY JESSESIONID VARCHAR(100); 

-- Microsoft SQL Server
ALTER TABLE CRASHPROTECTION ALTER COLUMN JESESSIONID VARCHAR(100);
ALTER TABLE REQUEST ALTER COLUMN JESESSIONID VARCHAR(100);
ALTER TABLE REQUEST_STATUS ALTER COLUMN JESESSIONID VARCHAR(100);
ALTER TABLE REQUEST_STATUS_REPORT ALTER COLUMN JESESSIONID VARCHAR(100);

6. Restart the FusionAnalytics Service/Daemon

This completed the procedure. You can now insert data with JESESSIONID values up to 100 characters long without truncation errors.