This article explains Data Catalogs and their use with host-based data sets.
In order to understand and use Catalogs, it's helpful to review the basic premise of SSProtect, the system of host/ endpoint :Foundation Client instances coordinating secured access to and management of protected application data using patented cryptographic offloading techniques instantiated by KODiAC (Cloud) Services. This platform of Unified Data Protection and Management Services protects sensitive application data from today's threats while maintaining application and infrastructure flexibility.
This premise primarily focuses on a 1:1 relationship between a managing host software component and a document, data file, or email message. When managed content is accessed, the :Foundation Client detects end-user interaction then applies policy-based security primitives that seamlessly integrate with existing application workflows while also offering a variety of unique protective services not otherwise available, perhaps even when integrating, "best of breed" capabilities from multiple point-products.
Catalogs represent an adaption of these fundamentals to host software that works with host data sets, often represented by a large number of corresponding data files accessed as a discrete whole. Catalogs aim to reduce the complexity of independently managing these individual items without compromise.
Catalogs for Practical Use
Consider a scenario where a managing/ editing software application uses several data files - or even hundreds of files - at the same time, rather than acquiring content from a single user document data file. Though the :Foundation Client supports these cases, the resulting overhead of individually processing each file creates results that are no longer suitable. Consider the resulting impact to performance, the practical use of fine-grained auditing replicating the same activity for hundreds of files, the automated execution of :Recover data backup/ restore for hundreds of MBs of (often-times static) data, and so on.
The use of SSProtect with such applications becomes impractical.
Data Catalogs allow you to define relationships between managing applications and large sets of data distributed across many individual files, applying service primitives to the transaction as a whole. This retains effective protections with practical instantiation of related system services.
IMPORTANT: Currently for Static Data Content
Catalog access does NOT currently retain changes made to individual data files while securely managed (accessed). This will change with ongoing adoption and feedback, though current support use is limited to STATIC DATA CONTENT.
Catalog Example: DICOM Image Files
Consider MRI data in DICOM file format, sometimes offered to medical patients on a CD-ROM:
This folder structure, copied directly from a CD-ROM to the C:\sImages folder, contains two studies for one patient distributed across 289 files in \DICOM subfolders (with over 130MB of data). The studies and associated images are indexed by information in the DICOMDIR file contained in the root folder. This resides alongside the \eFilmLite folder that contains a viewing application that can be executed without installation and/ or additional host-specific configuration.
When using an SSProtect Data Catalog to protect the associated data, eFilm Lite usage endures an additional 3.6 seconds of load time (on a reasonable business laptop against the USWEST KODiAC Server Set) while SSProtect applies protection and management capabilities to the 130 MB of data.
Ongoing end-user image browsing yields performance results nearly indiscernible from ordinary use with plaintext DICOM files - though as expected, the :Foundation Client maintains secure isolation of plaintext content against malicious intruders looking to, "offload decrypted data while it's in-use". In our example, this is critical since image browsing and rendering requires high-speed access to image data.
The remainder of this article uses this example to guide you through Catalog configuration and use while explaining differences between its' operation and the more typical 1:1 application/ document use case.
Catalog User Interface
To create a Catalog, you define the relationship between a Managing Application and target data files, at which point SSProtect applies protection for managed use. The :Foundation Client utilizes the Bulk Conversion interface for these proceedings, though Catalogs must be enabled for your Account by a Privileged User in User Administration (Catalogs are always enabled for Individual Accounts) else the managing controls will be hidden.
Navigate to the Bulk Conversion dialog using the SSProtect context menu (of the same name) available from the notification tray of the taskbar:
Choose Add then, when presented with the Folder Browsing dialog, navigate to the Catalog Target which is the folder that holds the associated application data and, in some cases, the associated application itself (more on this below). In our example, this is the C:\sImages folder.
If your Account is configured to use :Recover, you will be prompted with a Notice indicating that content will not be stored in or managed by the KODiAC :Recover Archive:
If you choose No or Cancel, the operation is aborted. At present, Catalogs are not associated with :Recover and not available for Backup/ Restore or Disaster Recovery services.
Choose Yes and the :Foundation Client will then, by default, attempt to auto-discover recognized DICOM relationships in the Catalog Target to fill in Data Catalog details for you:
Choose the Catalog Type from the dropdown, then click Discover and SSProtect will search the Catalog Folder to find and configure the most appropriate settings. Remember that this is automatically carried out for you when the Data Catalog dialog is first presented. Also note that, at present, SSProtect can only discover specifics for DICOM Image, "repositories", as shown. This will be expanded in the future.
Catalog Data File Retention
Once a Catalog is protected, SSProtect does not need individual ciphertext Data Files (in the Data Path) to securely access content: Protected Catalog content is managed in another fashion. Check Rem DataFiles (checked by default) to reduce the resulting size to an amount only slightly larger than the original plaintext size.
Note that, if/ when you later Remove protections, Data Files will be restored with their original plaintext content.
If however you decide NOT to remove ciphertext Data Files, you can uncheck Rem DataFiles and all resulting ciphertext files will remain present (in place of the original plaintext files). This results in a protected Catalog that's a little more than twice the size of the original plaintext.
An SSProtect Catalog is defined by the three distinct components shown in the Data Catalog dialog, with straightforward controls to make any necessary adjustments. These definitions are described below along with another optional setting, Val Signature. Refer to the section at the end of this document, Catalog Definition Requirements, for a list of current requirements associated with these definitions.
At present, SSProtect Catalog operation monitors a single data file, the Trigger File, for access by the Managing App before it interrupts proceedings to apply security primitives and secured access to managed content. In most cases, the Trigger File is some sort of data index or project definition that tells the Managing App how to load associated data content. In our example, this is the noted DICOMDIR file.
Data Path (currently for Static content ONLY)
The Data Path tells SSProtect where to look for data that should be encrypted/ protected. The contents of this folder, and all subfolders, will be protected once Catalog configuration details are entered and verified. Note that Data Path content should only include information used by the Managing App.
CRITICAL: Data Path content must be STATIC: secured access WILL NOT RETAIN CHANGES.
The Managing App is the executable that will ultimately be used to access Trigger File content before reading information contained in the Data Path. This deviates from the existing mechanism SSProtect uses when managing document/ data files, which relies on default application mappings for file types.
Val Signature, checked by default, determines whether or not SSProtect verifies the digital signature of the Managing App before protecting content in the Data Path. When checked, and if the Managing App does not carry a valid or trusted digital signature, Catalog creation will fail.
This is helpful if/ when your Managing App executes in an elevated state - in this case, SSProtect does not verify its' signature when, "triggered" (though will in all other cases if/ when required by your Account's Policy).
If this presents additional challenges, contact your SSProtect Administrator and/ or DefiniSec Support as noted at the end of this article.
Finalizing Catalog Configuration: Bulk Protection
Once you have finalized Catalog definitions noted above, choose OK to proceed. If (and as required) signature verification succeeds and all other Catalog requirements are met (see below), the software will then finish with a Bulk Protect operation for content in the Data Path folder and all its' subfolders:
Viewing Catalog Details
After a Catalog has been successfully created, you will see the resulting association in the Bulk Conversion dialog as shown below:
The, "S" column reflects, "N" for, "Native" or, "S" for, "Shared": As you peruse File Explorer content, you may discover other Catalogs created by other SSProtect Accounts. Enumeration in the Bulk Conversion Catalog Detail List does not imply access rights, :Collaborate details govern related access Policies. Use Refresh to update the Detail List (foregoing the need to close/ re-open to see discovered entries).
Select a Native Catalog from the Catalog Details List, then choose Modify. You can at that point change the Managing App to an executable (.exe) different from that already configured. Use Discover or Browse to affect changes, then OK. Changes go into effect immediately. If there are problems applying changes to Filtering, you will be prompted to Refresh Login. If you choose not to do so, you will see the SYNC FILTER status text underneath the OK button indicating that the Filter is not synchronized with the configuration. This state, when present, is also displayed if you navigate to Managed Files/ Restore.
You can also check Rem DataFiles (if/ when not previously checked) to remove Ciphertext Data Files from an existing Catalog. This reduces the resulting Catalog stored data size by nearly half.
Val Signature operation is as noted in the related text above.
Remove (/ Release) is slightly different than may be expected: With Native Catalogs, content in the associated Data Path will be Released from ongoing SSProtect management. This is carried out by Bulk Release execution (and as expected, you will see the status progress through related operation). This is referred to as Catalog Remove/ Release (to differentiate from List removal, below).
Note that, in the case where you've elected not to include Ciphertext Data Path files (Rem DataFiles, above), thereby reducing your resulting Catalog size, Remove (/ Release) will reconstruct the plaintext source files for you.
Remove (from List) with a Shared Catalog operates differently: Though the configuration is removed from your Account, and thus from the Catalog Detail List, protection isn't Released. In fact, you cannot Release protections for shared Catalog content using normal Bulk Release. You can however manually delete Catalog Data from storage, then execute Remove if the resulting Detail List does not reflect your changes.
Removed Catalogs and the Managed Files/ Restore Hostlist
SSProtect maintains state for managed content even after protection has been Released, as observed by the Hostlist's (Released) state for such items. This can be managed with the Clean or Remove facilities as described in the article, Managing Host Data.
Catalogs work a little differently. In general, the Trigger File is used to represent state, operations, and activity related to a Catalog as a whole. And though you can use the Hostlist's Cat Details to list the Index and Data Files, the Hostlist will not present them as (Released) after Remove/ Release operation. Instead, use the state of the Trigger File, which independently presents with (Released) and other Exception States.
Note that you can, as a result, Remove a Shared Catalog and re-discover it such that it, "re-appears" in your Detail List. Re-discovery is suppressed while using the same SSProtect Login Session as that used for Remove, however after Refresh Login and File Explorer perusal of the same Shared Catalog, the target association will re-appear.
Once a Catalog is configured and/ or discovered with an active SSProtect Login Session, and so long as your Account has access to associated content (using standard :Collaborate policies), subsequent execution of the associated Managing App will, "trigger" protected access to Data Path details.
In our example, the eFilm Lite viewer preloads all 289 images, though SSProtect interrupts this proceeding to apply security primitives and resulting In-Place Encryption/ Protection for ongoing access. If your Account is configured to use a 2FA token that requires physical acknowledgement, you will be prompted (once) to do so before content is accessed, converted, then presented in isolated form for ongoing use. As noted, on a typical business notebook computer, this introduces and additional 3-4 seconds of loading delay. SSProtect presents a Wait Dialog with associated status text, and depending on the amount of data in the Data Path, you may also see SSProtect Progress updates in the lower right corner of your Desktop.
Of course, and as expected, when you close the Managing App, ciphertext content replaces the otherwise inaccessible plaintext Data Path content before files are, "released". This takes a couple seconds though executes in the background, without displaying status or progress information.
Catalogs and Component Services
Component Services apply to Catalogs in a fashion slightly different from standard document/ data files. In general, a Catalog maintains SSProtect Component Service integration though in some cases limits activity to the Trigger File. This is the case when the applied activity would be repetitive.
For example, when first accessing a protected Catalog, :Assess Auditing will record activity specific to the Trigger File rather than repeating the audit entry 280+ more times for each individual Data File (for our example, above).
In general, Catalogs relate to Component Services as follows:
- Component Service features apply to a Catalog Trigger File at all times, as expected - with the noted exception of :Recover
- Identification, Authentication, and Authorization (:Access:/ :Confidential) primitives always apply to all aspects of the Catalog, including the Trigger File and individual files in the Data Path
- Component Service features OTHER THAN :Access/ :Confidential apply to Catalog Data Path files ONLY when the Catalog is created AND when the Catalog is Released (unprotected - Remove/ Release) - with the exception of :Recover which is at present not applied in any case
- Catalog Data Path files retain In-Place Encryption/ Protection capabilities, isolating in-use access to the authorized Managing App (in fact, the entire subfolder is blocked - as are individual files)
- :Confidential uses a modified form of re-protection for performance, applied to the Index and Data Files. This approach does not, at present, maintain any changes made after accessing Catalog content. For details, contact Support.
- Catalog Data Path file activities are not generally exposed to Component Service logic, but are instead represented by the Trigger File. In some cases, the Catalog Index, _catalog.ssp, is exposed to Component Service logic (though never to :Recover).
:Collaborate Data Sharing
Secure access to shared Catalogs is governed by :Collaborate Sharing Policies that can change over time. This does not impact the presence of a discovered Catalog in the Bulk Conversion Catalog Detail List. For more information, refer to the article, Protected Data Sharing.
:Assess and :Respond
Catalog creation and removal Converts individual Data Path files, and details for these activities are included in :Assess proceedings and thus available in related Reports and dependent services. For more information, refer to the article, Acquiring Data Access Reports.
Catalog access also Converts individual Data Path files, however these details are NOT included in :Assess proceedings since they are in some respect redundant (given the nature of accessing the Catalog as a whole). Trigger File and Catalog Index proceedings are, however, utilized and processed by :Assess.
As you may expect, :Respond Analysis will as a result provide Catalog Disclosure Risk Insight due to the inclusion of Trigger File proceedings. For more information, refer to the article, Definitive Disclosure Risk.
:Recover and :xRecovery
Catalog Data Path file content can be relatively large, hundreds of MBs if not more. Because of the impact to resources and common reality of static Catalog content, :Recover does not retain associated data for future Restore, and as a result Catalog content is not a part of :xRecovery Disaster Recovery operations.
This can, of course, change in the future based on needs and adoption.
Note that, if you create a Catalog with :Recover enabled, you will be prompted with a warning.
As noted elsewhere, and because :Recover is not currently utilized when accessing Catalog Data Path files, CHANGES TO CONTENT WILL BE DISCARDED.
File Management and :Respond Remediation
The Managed Files/ Restore details reflect Catalog details and proceedings, though by default Catalog Data File Hostlist enumeration is suppressed. Use the Hostlist Cat Details checkbox to toggle enumeration of all Catalog Data Files.
Because :Recover is not used, Versionlist (and Archivelist) details for Catalog Data Files will not be present, and in fact will not enumerate individual 0-length Versions typical of standard content. The Catalog Trigger File, however, will maintain Version-based insight, though as expected at 0-length due to the forced Optimized Offloading Operating Mode.
Finally, Catalogs include a protected _catalog.ssp file, which presents independent from Cat Details though does not contain Version-based instances for each access event.
For more details, refer to the article, Managing Host Data.
Catalog Definitions and Technical Insight
This section further explains requirements limitations for this early release of Catalogs while also providing further insight into the support files SSProtect creates to work with and manage the protected data set.
Catalog definitions are independently managed for each target Environment. For Catalogs, the Environment is scoped by the unique combination of a target host computer and active Windows User Profile. When using your Profile with multiple Windows Users on a single host computer (or different host computers), you may as a result observe different results in the Catalog Detail List of the Bulk Conversion dialog.
Catalog Definition Requirements
Catalogs were first released at the end of the 2nd Quarter 2020, and as such adhere to a fairly restrictive set of configuration requirements as adoption and experience provide the insight required to enhance operational capabilities. For these - and a variety of other reasons beyond the scope of this article - the following relative requirements apply to the defining aspects of a Catalog (and are enforced where and when possible, during configuration):
- The Catalog Target (when first browsing for Catalog content) must encapsulate the Trigger File and Data Path, though the Managing App can be located anywhere
- The Catalog Target must be independent of the Default (Overlfow) Folder in every way (cannot be inside or outside of it)
- The Trigger File must reside, "outside" the Data Path (it cannot be in a Data Path folder)
- The Catalog Target must not overlap the Catalog Target of any other known Catalog
Relative Catalog Limitations
With respect to, "typical" 1:1 document/ application associations and SSProtect Component Service behavior, the following set of Limitations represent those areas where operation deviates from what would otherwise be achieved by applying SSProtect to all files, independently, and/ or for those areas where practical use of Catalogs requires additional adjustments to improve usability.
- A Catalog is only sufficient for managing STATIC DATA: CHANGES made to Data Path file content WILL BE DISCARDED after access to managed plaintext and when converted back to ciphertext.
- You cannot access more than one protected Catalog at a time - an attempt to open a 2nd Catalog, concurrently, will fail.
- You cannot Add/ Remove a Catalog while securely accessing a protected Catalog. There reverse also holds true: You cannot access a protected Catalog while Adding/ Removing one.
- When you Login to SSProtect, if the Trigger File for a configured/ discovered Catalog is not present, cannot be accessed (for any reason), and/ or cannot be verified as an SSProtect'd item, the Catalog will be removed from the Bulk Conversion Catalog Detail List. Re-discovery requires Refresh Login and adjustments to the inhibiting dynamic.
- You cannot edit or modify the scoped content (Data Path details) of a Catalog without Remove/ Release and re-creation of the modified content.
- At present, there is no way to Restore Catalog content, since :Recover ignores Catalog details.
- You must establish a Login Session before attempting to access a Catalog - if the Managing App accesses the Trigger File and you don't already have an established SSProtect Session, unlike the case with other managed items, SSProtect will not prompt you for Login credentials and proceed: Access will simply fail (though you can close the Managing App, Login to SSProtect, and retry).
The following Catalog files are created, in the Catalog Target folder, to govern secured access:
These files assist in managing the relationships between a Trigger File, Data Path, and the Managing App that make up a Catalog.
_catalog.ssp, a large file, is not submitted to :Recover for secure storage, in any circumstance. The detail file, _catdet.ssp, will in the future likely be removed and embedded into the Profile and/ or KODiAC user-specific data for further isolation. This file does not contain sensitive information, but its' presence is perceived to be unwieldy.
Future Catalog releases will perhaps remove the presence of both files, though for now it's important to know that these must be copied when a Catalog is moved from one location to another.
Finally, as noted above, a Catalog content cannot be changed on the fly - it must first be Removed/ released then re-created with modifications. The Managing App however can be changed. Future releases will extend overall flexibility.
You can search this site for more information on various topics, or use this link to submit a specific request. You can also send email directly to firstname.lastname@example.org, and our staff will respond to your needs as soon as possible.
In the meantime, don't forget to check out our primary website and Insights columns for information on current trends, security topics, and how our technologies relate.
This article was updated w/ v10.7.1 of the :Foundation Client