FusionAnalytics Rev. 2.0.8

Release Notes

Status: 03/Dez/2018

Welcome to FusionAnalytics !
We appreciate your feedback. Please use the web form or send mail to:

support at fusion-analytics.com

These Release Notes describe what is contained in this release, provide
late-breaking news, and list additional documentation for the software.
For additional information on FusionAnalytics, please visit our website at:

https://www.fusion-analytics.com

The help for this product is now online, where it can be constantly updated
and refreshed. You can find it here:

https://intergral.atlassian.net/wiki

For known issues and further support, please see the following web pages:

https://www.fusion-analytics.com/fa/faq.cfm
https://www.fusion-analytics.com/fa/support.cfm

———————————————–

RESOLVED ISSUES 2.0.8

RESOLVED ISSUES 2.0.7

RESOLVED ISSUES 2.0.6

RESOLVED ISSUES 2.0.5

RESOLVED ISSUES 2.0.4

RESOLVED ISSUES 2.0.3

RESOLVED ISSUES 2.0.2

RESOLVED ISSUES 2.0.1

RESOLVED ISSUES 2.0.0

Introduction

This technote will provide readers with more information about a common licensing issue which is caused by the use of an Anti Virus software in the working environment.

More details about the issue

We have encountered a number of cases where users are not able to activate a valid license key with FusionAnalytics.
Even if the license key is valid and has not been expired, the users are unable to activate it.
In order to make sure that you encounter the same issue, please check out the log files created by FusionAnalytics and search for the following error message. See screenshot below.

The log files can be found in the following directory; C:\FusionAnalytics\Server\logs.

The development team figured out that some certain anti virus software like Nod32, Avira and AVG prevent FusionAnalytics to work correctly.The issue appears to be that the BlazeDS communication between the FusionAnalytics client and the FusionAnalytics DataServices (FADS) is broken by the use of the anti virus software.

Solution

In order to solve the issue and activate the license key successfully, please make sure that any anti virus software is disabled during the activation process.

If you have further questions or issues, please feel free to get in touch with the FusionReactor support team on

Description
The Data Volume Management job in FADC goes through the process of updating indices and stats.
In previous versions of FusionAnalytics (2.0.1 and earlier), a command called sp_updatestats was used to do this.
However, this command requires the database user to have the sysadmin server role enabled.

You may feel that it is too much of a risk to enable the sysadmin role. If so, you can download an alternate Data Volume Management script, which uses a different command and does not require the sysadmin server role.

Before Starting Please Note
As an alternate to the sysadmin server role, the user will require the db_owner database role enabled for the FusionAnalytics database. This is the minimum that is required for FADC to run all of it’s jobs correctly.

To enable the db_owner role:

1. Open Server Management Studio.
2. Login as an existing user with system administrator privileges.
2. Navigate to Your DB -> Security -> Logins.
4. Right-click the user you wish to update permissions for and select Properties.
5. Select the “User Mapping” page.
6. Select the Database you wish to map the user to
7. Ensure ‘db_owner’ is checked in the list of database roles.
8. Press OK to save the changes.

Instructions
1. Download this script
2. Navigate to the folder: FusionAnalyticsServerdatafadcApplicationsmyApplication
3. Replace the “manageDataVolume.groovy” file in this folder with the downloaded script.
4. Restart the FusionAnalytics server.

After this, whenever the Data Volume Managment job runs, the alternate script will be used.

Cleanup obsolete request data from database

Description
Due to a bug in the Data Volume Management job shipped with FusionAnalytics 1.0.4, parts of the request data in tables REQUEST_AMF and REQUEST_PARAM are not deleted from the database.
To cleanup your database and remove the obsolete data you can download this script and run it on your FusionAnalytics database server.

Before Starting Please Note

• You will require full access to your FusionAnalytics Database Server and all permissions such as create and delete tables.
• You will need to specify the database name that is used by FusionAnalytics in the script (see instructions below)
• This script might take a long time to execute. This is dependent on the following factors:
  • The size of your database.
  • The performance of the hardware where the database is installed.
  • The number of rows that became obsolete.

Example: It will take between 10 to 20 minutes on an environment that has a 200GB database that is installed on a standard HDD and has 40 Million (40,000,000) obsolete rows

Instructions

1. Download this script
2. Open SQL Management Studio
3. Login as an existing user with system administrator privileges.
4. Navigate to File -> Open -> File
5. Navigate to the folder where the script is located and select it
6. At the top of the script enter your FusionAnalytics database name into the square brackets “[]”
   If your database name is “FADB” then it should look like this:

   USE [FADB]
   GO
   
   declare @totalRows int, @rowsToKeep int
   ...

7. Click the Execute button

When the script has finished you should see the following messages:

(_n_ row(s) affected)
Caution: Changing any part of an object name could break scripts and stored procedures.

(_n_ row(s) affected)
Caution: Changing any part of an object name could break scripts and stored procedures.

Where n is some number of rows

This technote provides access to the FusionAnalytics 2.0 Documentation as a single PDF to download and read offline. Please note that the PDF is generated periodically and may not reflect the latest changes to the online documentation.

FA20-170113-1139-7.pdf

“User does not have permission to perform this action” when running application jobs.

This error can occur when attempting to perform FADC application jobs on your database without sufficient permissions. FADC requires ‘sysadmin’ permission on the database to be able to perform its jobs correctly.

To edit the permissions of an existing user with Microsoft SQL Server Management Studio, you can:

1. Open Server Management Studio.
2. Login as an existing user with system administrator privileges.
3. Navigate to Your DB -> Security -> Logins.
4. Right-click the user you wish to update permissions for and select Properties.
5. Select the “Server Roles” page and ensure ‘sysadmin’ is checked.
6. Press OK to save the changes.

When viewing FusionAnalytics reports which have been delivered via email, you may see a request to log in to your FusionAnalytics Data Services server.

This happens because the images which comprise part of the reports are served directly from the FADS system itself.

If you are seeing a request to log in, you have not obtained a cookie for the FADS server on the address from where images are being served.

There are three solutions to this problem.

Log in.

When the login dialog appears, log in as a FADS user who is able to view the images.

Adjust the security settings of the application

You can set your FADS application not to require authentication to view images.

Warning: If you disable this protection, anyone will be able to view these report images, regardless of whether they are a valid FADS user or not.

1. Navigate on your server to FusionAnalyticsServerwebappsfadsWEB-INFclassesApplicationsYOUR_APP_NAME and open application.xml with a text editor. This editor must be capable of reading and writing UTF-8 Unix files. Windows Notepad is not able to do this – we recommend Notepad++.
2. In this file, locate the ApplicationWebServer block and carefully remove the Path whose path attribute is /_Persistent.
   
   
   
   You should be left with:

       <ApplicationWebServer>
          <PathRules>
             <Path path="/configuration">
                <Protected/>
             </Path>
          </PathRules>
       </ApplicationWebServer>
   

   
   

3. Restart the application by logging in to the Data Services webapp (usually located on port 8400), and restarting the data service itself using the Restart button.

Caveat: If the application is upgraded or reinstalled, this procedure must be repeated.

Change the domain name used to serve image links

The problem can also be solved by changing the host name used to serve image links. If you ordinarily log in to the FusionAnalytics Data Service webapp using – for example – http://fred.foo.com:8400/, then you should already have an authentication cookie for this domain. If the system has several aliases, images may be being served from a different domain name and the login dialog will appear. If this is the case, the domain name used in links can be changed:

1. Log in to the Data Services webapp (usually located on port 8400). Open the System section and click on the Server Settings entry.
2. Under the settings HTTP Server Host and HTTPS Server Host, change the value to the domain name you usually use.
3. Click Update Server Settings
   • This will cause generated links to use the new domain name, for which you already have an authentication cookie, and the login dialog should no longer appear.

FusionAnalytics Rev. 1.0.4

Release Notes

Status: 27-09-2012

Welcome to FusionAnalytics !
We appreciate your feedback. Please use the web form or send mail to:

These Release Notes describe what is contained in this release, provide
late-breaking news, and list additional documentation for the software.
For additional information on FusionAnalytics, please visit our website at:

https://www.fusion-analytics.com

The help for this product is now online, where it can be constantly updated
and refreshed. You can find it here:

https://intergral.atlassian.net/wiki

For known issues and further support, please see the following web pages:

https://www.fusion-analytics.com/fa/faq.cfm
https://www.fusion-analytics.com/fa/support.cfm

———————————————–

RESOLVED ISSUES 1.0.4

Background

FusionAnalytics is a platform for simplifying complex data analysis. Part of its task is managing the storage requirements of analysis data. Typically, first installs use a “the more the better” mindset. However, for busy or poorly written systems, they can often generate a lot of data.
This technote article discusses where the data comes from, what data is needed and the different techniques to reduce the data volume requirements.

Where is the data?

FusionAnalytics includes the “Analytics for FusionReactor” data analysis application. In a typical scenario, data is generated by FusionReactor, stored in log files, then sent using a HTTP interface to FusionAnalytics. FusionAnalytics stores the files, then after processing them into a DB will archive them.

This gives us several points where data needs to be stored, and options for managing that incoming & stored volume.

Reducing incoming data

The first logical option is to look at reducing the amount of data coming into FusionAnalytics. The largest data volumes come from two log sets – the request logs and the JDBC logs. The request logs contain entries for each web application request hitting the server. The JDBC logs contain entries for every JDBC (SQL) query executed. Typically each request will contain one or more JDBC query entries and therefore the JDBC log is typically the largest.

Only log slow JDBC queries

Once you’re comfortable with your FusionReactor/FusionAnalytics setup, a simple method to reduce the data is to only log slow JDBC queries – because typically fast queries aren’t of so much concern for analysis.

How To?

To enable this, login to FusionReactor, click JDBC->JDBC Settings. Then under “JDBC Logging (Log File)” enter a number in the “Only queries slower than: (ms)” box. Whilst every system is different, values of 50 or 100ms should be a good starting point.
Documentation: https://intergral.atlassian.net/wiki/display/FR452/JDBC+Settings

• For: Simple to implement
• For: Requires less disk space on application server
• For: Shorter data transfer time & less bandwidth used between FR & FA
• For: Data still available in FR web interface
• For: FA log file data volume requirements lowered
• For: FA DB data volume requirements lowered
• Against: Can hide issues with high volume of fast queries
• Against: Fast queries won’t be logged to disk, even on the application server.

Don’t automatically send JDBC data to FA

An alternative (or even additional) option is to not automatically send JDBC log data to FA from FR. Doing this keeps the (typically larger) JDBC log data on the application server only. This helps by keeping the FA log archive & DB data volume requirements lower.

How To?

To enable this, login to FusionReactor, click Plugins->Active Plugins. Then under the “FusionReactor Log Rotator” plugin, click “Configuration”.
Change the “JDBC Log” option from “Transfer and Archive” to “Archive only”.
Documentation: https://intergral.atlassian.net/wiki/display/FR452/FusionReactor+Log+Rotator+Plugin

• For: Shorter data transfer time & less bandwidth used between FR & FA
• For: Data still available in FR web interface
• For: FA log file data volume requirements lowered
• For: FA DB data volume requirements lowered
• Against: Requires manual action to get FA to analyze JDBC data.
• Against: Can result in client wanting to keep log archive data on the application server for a longer period of time. (eg trade-off FA server log/DB space for application server space)

Reducing stored LOG files (on the FA server)

Approximately 5-10% of all FA data is required for a) log queue (incoming, queued & processing) b) log archive.

Reducing the storage requirements for incoming logs

Without actually altering the content of incoming logs, their size can be reduced by compressing them before they’re sent to FA. Although this is enabled by default, ensure it is enabled.

How To?

To enable this, login to FusionReactor, click Analytics->FusionAnalytics Targets. Ensure “Sent to targets by this instance (uncompressed)” is NOT selected.
Documentation: https://intergral.atlassian.net/wiki/display/FR452/FusionAnalytics+Settings

• For: Simple to implement
• For: Shorter data transfer time & less bandwidth used between FR & FA
• For: Data still available in FR web interface
• For: FA log file data volume requirements lowered

Deleting old archives

Old log archives can be deleted after processing. For auditing or rebuilding a broken DB, it’s possible to keep the processed logs for a period of time. If neither of these issues are a concern, you can minimize the volume of archived log files to be kept.

How To?

Login to FusionAnalytics:DataCollector (FADC). “Stop” the application. Click “Configure”. Under the “FusionAnalytics Processed Log File Management” header, enable the option and set how long you want to keep log files for. Alternatively, you can choose a data volume (disk space) based strategy. To keep a minimal set of logs, set the retention period to “1 day” / “1 MB”.
Documentation: https://intergral.atlassian.net/wiki/pages/viewpage.action?pageId=22479170#ApplicationConfigurationFADC-filemanagment

• For: Simple to implement
• For: FA log file data volume requirements lowered
• Against: Log files no longer available for auditing or should FA DB fail

Reducing DB size (FA server DB)

Deleting historical high resolution data.

Typically the value of the high resolution (/high detail) data degrades rapidly with time. EG it’s rarely useful to see the exact DB queries running on the same day in the previous year. However, the low resolution (/aggregate) data can in many way become more useful (eg capacity planning – show me a chart of how many DB queries I’ve run over the last year).
High resolution data takes a lot of DB space. Deleting it sooner (but keeping low resolution, aggregate data) can save lots of disk space.
EG: On a busy system receiving over 10,000,000 requests per day, changing high-resolution storage from 1year to 1week made a better than 98% DB size saving.

How To?

Login to FusionAnalytics:DataCollector (FADC). “Stop” the application. Click “Configure”. Under the “FusionAnalytics DB Data Management” header, enable the option and set how long you want to keep high/low resolution data for.
Documentation: https://intergral.atlassian.net/wiki/pages/viewpage.action?pageId=22479170#ApplicationConfigurationFADC-datamanagment

• For: Simple to implement
• For: FA DB data volume requirements lowered
• Against: Detailed data is only available for a shorter period of time

DB optimization

Whilst specifics are too detailed for this technote, it’s worth noting that your DB can likely be configured to optimize/return space to the host system. Some of these options would also benefit performance. For example, using our recommended SQL Server DB:

• Set the recovery model to “Simple”
• Set “Auto Shrink” to “True”

Further details

For more information including our installation & configuration service or product consulting, please contact