The Custom Validation feature is no longer a part of the AMF core as from version 5.x.x onwards. It has been extracted and published to a new project called AMF Custom Validator.
Just like any other validator, the AMF Custom Validator takes two inputs: data and contraints. Specifically, it takes the AMF JSON-LD output as data and a Ruleset (former Validation Profile) as contraints and returns a validation report.
Using the AMF Custom Validator
To get started with the AMF Custom Validator refer to the README file in the project repo.
To write a Ruleset, please refer to the: