Contents
How to create a custom classification and classification rule?
Optional. A regular expression that represents the data that’s stored in the data field. The limit is very large. In the previous example, the data patterns test for an employee ID that’s literally the word Employee {GUID}. Optional. A regular expression that represents the column names that you want to match.
How are classification rules used in Catalog Service?
The catalog service provides a set of default classification rules, which are used by the scanner to automatically detect certain data types. You can also add your own custom classification rules to detect other types of data that you might be interested in finding across your data estate.
How are categorical features compressed into fewer levels?
When categorical features contains many levels, it can be compressed into fewer levels using this parameter. It takes a list of strings with column names that are categorical. Categorical features with high cardinality are replaced with the frequency of values in each level occurring in the training dataset.
What are the thresholds for a data pattern?
Under Data Pattern, there are two thresholds you can set: Distinct match threshold: The total number of distinct data values that need to be found in a column before the scanner runs the data pattern on it. The suggested value is 8. This value can be manually adjusted in a range from 2 to 32.
When to add conditions to a rule based classifier?
In the general-to-specific approach, start with a rule with no antecedent and keep on adding conditions to it till we see major improvements in our evaluation metrics. While for the other we keep on removing the conditions from a rule covering a very specific case.
Which is the best method for rule based classification?
Sequential Covering such as CN2 Algorithm and RIPPER Algorithm are common direct methods for building classification rules. Take Ripper method as example. For 2-class problem, choose one of the classes as positive class, and the other as negative class, learn rules for positive class, and negative class will be default class.
What does coverage mean in rule based classifier?
Coverage: The percentage of records which satisfy the antecedent conditions of a particular rule. The rules generated by the rule-based classifiers are generally not mutually exclusive, i.e. many rules can cover the same record.