Protegrity has released an open-weights model for detecting personally identifiable information (PII) in text, together with a benchmark called PII-SCOPE. The model is a fine-tune of RoBERTa trained on synthetically generated text containing many types of personal data.

Psst!Add AlexTech.ai asPreferred Source on Google

The system identifies instances of personal data from the context around them and returns the position and type of each. Alternatively, it can return the input with all personal data redacted. The model, its script and the technical report describing its architecture and results are available on Hugging Face.

official image of A new, open-weights 278M PII detection model and reference benchmark R

Image: philterd.ai — https://philterd.ai/blog/auditing-our-pii-models-with-philter-scope/

According to the model card, the system reaches a typed recall of 0.910 at a precision of 0.902 on PII-SCOPE. The technical report gives slightly higher figures, 0.915 recall and 0.907 precision; the sources do not explain the difference.

The benchmark was built, according to its authors, to address problems common in public datasets generated by language models. Its documents are long, realistic and densely packed with many types of PII, which makes them harder than most public benchmarks.

The model supports English, Spanish, Portuguese, German, French, Italian and Dutch. It is released for research and non-commercial use, and its producer warns it must not be the sole safeguard for personal data.

One source also cites a training set of 1.2 billion synthetic text samples; that figure appears in only one of the sources consulted and is attributed to it.