Sophos Reporter Release Notes
Subscribe
Sophos Reporter 3.0.0.11 Beta (2019-02-22)
  • Fixed Settings | Diagnostics | Database page showing NaN% on disk usage when total is 0.
  • Service stop stage when installing now shows 'Stopping service…' instead of 'Extracting files…'.
  • JRE unpack stage in installer now waits for 'binjava.exe' to exist before proceeding to work around asynchronous file creation issues.
  • Dashboard engine now processes widgets at the correct interval.

Join the Fastvue Product Testing program to access Fastvue’s latest pre-release products.

Sophos Reporter 3.0.0.10 Beta (2019-02-15)
  • Added 'best_compression' option to Elasticsearch instance to reduce disk size of imported data.
  • Excluded certain URLs from the Search Term widgets and alerts to reduce auto-complete searches and other noise
  • Restyled the sub-report-type tabs to look more consistent with the rest of the app.
  • Fastvue Reporter now monitors disk space usage separately for both Reporter's data storage location and Elasticsearch's database location.
  • Disk usage warning/critical thresholds are configurable separately for storage and database, either as percentages of total space or as fixed sizes. This is configured in the Settings.xml file with the following settings: 
     <Item Name="DiskSpaceStorageLowSpaceWarning" Type="System.String, mscorlib" Value="15%" /> <Item Name="DiskSpaceStorageLowSpaceCritical" Type="System.String, mscorlib" Value="5%" /> <Item Name="DiskSpaceDatabaseLowSpaceWarning" Type="System.String, mscorlib" Value="15%" /> <Item Name="DiskSpaceDatabaseLowSpaceCritical" Type="System.String, mscorlib" Value="5%" />
  • Added email notifications for when disk usage threshold limits are reached.
  • Added cleanup task that automatically runs when either storage or database disk usage thresholds switch to warning or critical.
  • Added a notification message to site header when database disk space is critically low.
  • Added a link to Settings | DataStorage to the Storage row of the Database status page in Settings | Diagnostics | Database.
  • Low disk space warning/critical status changes the database status colour in Settings | Data Storage and Settings | Diagnostics.
  • Elasticsearch disk status critical now only stops import but continues to allow queries.
  • Elasticsearch status will be changed to red if the 'critical' disk space threshold is passed.
  • An email will be sent to the System Notifications email address (specified in Settings | Email) when the disk space status changes.
  • Legacy data migration process now applies data retention policy to legacy data stores.
  • Migration status now shows if it is rereading up to its last position for the current store (shows 'Preparing migration for (date) (xx%)').
  • Elasticsearch statistics are now cleared when the connection is lost so that UI doesn't show stats while disconnected.
  • Added exponential backoff (1s-300s) to Elasticsearch automatic restart in case of unexpected Elasticsearch termination.
  • Improved Elasticsearch restart logic when Elasticsearch is unexpectedly terminated.

Join the Fastvue Product Testing program to access Fastvue’s latest pre-release products.

Sophos Reporter 3.0.0.9 Beta (2019-02-05)
  • Fixed issues when filtering on Productivity groups.
  • IP fields in Elasticsearch can now be filtered using StartsWith/EndsWith/Contains and their inverse variants.
  • Fixed issue filtering on blank values.
  • Series 'Other…' is now forced for charts where it makes sense to do so.
  • The installer now removes existing JRE before installing new JRE version, and when uninstalling the product.
  • Elasticsearch self-managed mode now supports custom instance location using the DatabaseElasticSelfManagedPath in Settings.xml which falls back to (datalocation)/Data.elastic if unset/blank.
  • Elasticsearch self-managed mode now prefers JRE in instance path first.
  • Updated Elasticsearch package to 5.6.14.
  • Updated JRE package to OpenJDK OpenJ9 1.8u192.
  • Elasticsearch interface now updates status to 'Warning' or 'Failure' if Elasticsearch unexpectedly switches to yellow or red status during normal operation instead of changing colour but continuing to display 'Operational'.
  • X-UA-Compatible header no longer causes an error about duplication.

Join the Fastvue Product Testing program to access Fastvue’s latest pre-release products.

Sophos Reporter 3.0.0.8 Beta (2018-12-20)
  • Some fields now support case-insensitive filtering, such as Users.
  • Fixed forced lowercasing of LDAP display names.
  • The installer now correctly sets the IIS App Pool to .Net 4.0.
  • Increased Elasticsearch self-managed startup timeout to 5 minutes from 2 minutes.
  • Increased Dashboard update interval to 15 seconds from 5 seconds.

Join the Fastvue Product Testing program to access Fastvue’s latest pre-release products.

Sophos Reporter 3.0.0.7 Beta (2018-12-17)
  • Some fields now support case-insensitive filtering, such as Users.
  • Fixed forced lowercasing of LDAP display names.
  • The installer now correctly sets the IIS App Pool to .Net 4.0.
  • Increased Elasticsearch self-managed startup timeout to 5 minutes from 2 minutes.
  • Increased Dashboard update interval to 15 seconds from 5 seconds.

Join the Fastvue Product Testing program to access Fastvue’s latest pre-release products.

Sophos Reporter 3.0.0.6 Beta (2018-12-17)
  • The Overview and User Overview Reports now have three sub-report-types – Internet Usage, IT and Network Security, and All Usage.
  • Improved Browsing Time calculation.
  • Fixed filter StartsWith, EndsWith and Contains operators and their inversions.
  • Usernames are now forced to `DOMAINusername@domain` casing before being indexed in Elasticsearch.
  • Added three tabs to Settings | Diagnostic: Logs, Resource Usage, Database.
  • Moved database status display to Settings | Diagnostic | Database.
  • Added CPU, RAM, and Disk charts, as well as Database charts for CPU and RAM to Settings | Diagnostic | Resource Usage.
  • Improved database status information provided by API, details about Elasticsearch's nodes and shards are now provided in a structured form and displayed in Settings | Diagnostic | Database.
  • Changed the layout of text on the Settings | Diagnostic page and updated the log location to show the new location of the logs.
  • Added database interface status display in Settings | Data Storage.
  • Errors during record import will now queue the failed record to be imported again with the next import batch.
  • New API:
    – Storage.GetDatabaseStatus – Gets the current database/Elastic status (also available via Storage.GetStatistics().Database).
  • Elasticsearch startup timeout is now configurable via setting `DatabaseElasticSelfManagedTimeout` in Settings.xml (default 120s).
  • Elasticsearch interface now checks the Elasticsearch version on connect.
  • Elasticsearch interface now stores scripts in Elasticsearch rather than passing the script source with each query.
  • Elasticseach interface now waits for Elasticsearch to complete index recovery before loading indexes, pushing scripts, or accepting query/index requests.
  • Fastvue Reporter now uses health status to automatically attempt to reconnect to Elasticsearch on connection failure and reload indexes and push scripts.
  • Fastvue Reporter now reloads indexes and pushes scripts if a health check fails.
  • Fastvue Reporter now disconnects/reconnects to Elasticsearch completely if no health check has succeeded within the last 60s.
  • Index settings are now updated in parallel with error checking and retry.
  • Stored scripts are now stored in parallel with error checking and retry.
  • Restructured stored scripts push logic to precache all of the scripts before storing them in Elasticsearch.
  • Elasticsearch node stats update no longer logs errors if no response is received.
  • Elasticsearch start now scans for the major version of Elasticsearch in use and sets command line arguments to java appropriately.
  • Elasticsearch now logs the full command line used at Verbose level.
  • Elasticsearch wildcard/prefix queries are now added to the 'must' section of the query instead of the 'should' section.
  • Elasticsearch index deletion now also removes the cached date statistics.
  • Elasticsearch interface will now consider the instance started when the HTTP server is reported active.
  • Fixed GetDates in Elasticsearch interface not correctly checking connected status.
  • Elasticsearch interface now ignores 'yellow' cluster health status from Elasticsearch for 30 seconds after attempting to create an index.
  • Errors in index creation or bulk indexing will now force the index list to be reloaded to ensure consistency.
  • Updated Elasticsearch version to 5.6.13.
  • Data retention will no longer attempt to scan an empty list of dates.

Join the Fastvue Product Testing program to access Fastvue’s latest pre-release products.

Sophos Reporter 3.0.0.5 Beta (2018-11-19)
  • Added Filename field.
Sophos Reporter 3.0.0.4 Beta (2018-11-16)
  • Added support for Sophos XG's Application Log.
  • Rule field is now populated by looking up the meaning of the message ID in the Sophos log documentation.
  • Added Syslog over TCP support
  • Improved performance of report widget generation.
  • The appropriate sidebar item in Overview Reports now highlights properly when Report Drawer and Report Options are shown at the top of the page.
  • Added error handling to legacy data migration.
  • Improved filtering on aliased values such as Productivity and Departments.
  • Re-added Data Path setting to Settings | Data Storage page.
  • Removed version number from default Program Files paths in installers.
  • Added verbose logging for report widget generation timing.
  • User/Device statistics calculators no longer fail to read a file when duplicate names are present.
  • Diagnostic log archival now zips to a temporary file ({logname}.zip.temp) before moving to the final archive name ({logname}.zip) after the zip operation has completed.
  • Elasticsearch interface now only queries indexes related to the query by date filter (or if no date filter, queries all indexes).
  • Queries will now be retried if they fail due to '429 Too Many Requests' error.
  • Elasticsearch stats API call now has a timeout of 2s instead of 500ms.
  • Per-date statistics (record count and disk size) are now cached separately to database interface to allow for Elastic to lazy-load its index statistics without causing the appearance of missing or slowly loading data in Settings | Data Storage.
  • Elasticsearch interface no longer fails when connecting to an Elastic instance with indexes that do not follow the expected naming convention.

Join the Fastvue Product Testing program to access Fastvue’s latest pre-release products.

Sophos Reporter 3.0.0.3 Beta (2018-10-24)
  • Added Internet Usage, IT and Network Security Reports and All Usage Reports
  • Added support for Sophos XG's Malware, IPS, Sandstorm, and ATP events.
  • Rule is now set to Action if Action contains 'sandbox'.
  • The `pua` field is imported to ThreatName if the `virus` field is not present.
  • The `country` field is now imported.
  • UTM name is populated from the Sophos hostname section in the log.
  • Added new Block Evidence cases.
  • Removed 'Web request warned, ' string from start of the Rule field to better show the reason for the warn.
  • Added 'Uncategorized', blank categories and 'Categorization failed' to the 'Unassigned' Productivity group (Sophos)
  • Added 'Political Extreme / Hate / Discrimination' to the Unacceptable list (Sophos)
  • Added 'Quota' and 'Send to sandbox' Actions to the sample data / autocomplete info (Sophos)
  • Elasticsearch now creates a new index for each date. 
  • Improved data size calculation per date in Settings | Data Storage.
  • Added Data Migration feature to automatically migrate legacy Fastvue Reporter data stores (FVFS format) to Elasticsearch. Data migration process can be viewed in Settings | Data Storage
  • New APIs:
    – Storage.GetMigrationStatus() – Returns the status of the current migration task if any.
    – Storage.StartFvfsMigration() – Manually start the Data.Fvfs migration.
    – Storage.SetMigrationPaused(paused) – If paused is true, pauses the migration, otherwise resumes it.
  • Performance Improvements.
  • Improved Installer (error checking, bug fixes)
  • Fixed issue where the Time Analysis Widget would fail to generate correctly.
  • Fixed issue with Activity Reports not rendering.
  • Improved diagnostic logging.
  • Reporter now monitors Elasticsearch process status and restarts it automatically if Elasticsearch terminates or exits without being instructed to do so by Reporter.

Join the Fastvue Product Testing program to access Fastvue’s latest pre-release products.

Sophos Reporter 3.0.0.2 Beta (2018-09-26)
  • First working build of Reporter 4.0 with Elasticsearch

Join the Fastvue Product Testing program to access Fastvue’s latest pre-release products.