Benchmark Evaluation Report

PII2PW: Personal Information to Password Wordlists

A rule-based, training-data-free targeted password guesser for Chinese users
github.com/WangYihang/pii2pw · April 2026 · Comparative analysis against CUPP and six academic baselines

Abstract.We evaluate PII2PW, a rule-based targeted password guessing tool for Chinese users, against CUPP (the de-facto standard) and published results from six academic papers spanning CCS 2016 to S&P 2025. Using five standard profiles and a 200-record synthetic PII–password paired dataset modelled after real Chinese password patterns (Li, Wang & Sun, INFOCOM 2016), we measure generation performance, PII embedding rate, dataset hit rate, and Success Rate @ N. PII2PW reaches 97.5% coverage and SR@100 = 46.5% on this set, generating 6× faster than CUPP with zero training data, GPU, or external dependencies. Because the paired set is synthetic and shares its assumptions with the rule engine, these absolute values are optimistic and are not comparable with the academic baselines below — see the fair-comparison caveat in §6.

1.Key Results

76.5%
SR@1000
97.5%
Coverage / 200
2.4 M/s
Generation
6×
vs. CUPP

2.Generation Performance

We measure password generation throughput across five standard benchmark profiles — three Chinese (zh_full, zh_minimal, zh_medium) and two English (en_full, en_minimal). All measurements are wall-clock on a single CPU thread.

Table 1.Password generation statistics. Bold values mark per-row maxima. PII2PW generates fewer total candidates but at a substantially higher rate.
ProfileToolPasswordsTime (s)Passwords / s
zh_fullCUPP28,6470.072396,071
PII2PW17,1000.0072,382,811
PassLLM2,13344.45148
bopscrk19,6751.05618,623
zh_minimalCUPP5,2460.026202,408
PII2PW7,1980.0041,792,365
PassLLM2,17735.41961
bopscrk1,5760.9061,739
zh_mediumCUPP18,6660.059317,377
PII2PW13,2180.0043,197,065
PassLLM1,97334.05658
bopscrk25,1571.78714,079
en_fullCUPP22,4080.085262,185
PII2PW6,8870.0023,194,313
PassLLM2,10334.24461
bopscrk14,2060.98014,491
en_minimalCUPP9,5940.026364,389
PII2PW3,5810.0013,227,983
PassLLM2,18932.28268
bopscrk6720.873770
Figure 1a.Generation throughput (pwd/s).
Figure 1b.Total candidates generated.

3.PII Embedding Rate

Fraction of generated candidates that contain a fragment of the target’s personal information. Higher rates indicate more targeted generation. The Personal-PCFG study (Li, Wang & Sun, INFOCOM 2016) observed that 60.1% of the leaked 12306 passwords contain at least one type of PII.

Table 2.PII embedding rate by category. Bold values mark per-profile maxima.
ProfileToolNameDatePhoneAccountOverall
zh_fullCUPP27.1%4.6%0.0%7.3%33.2%
PII2PW25.8%23.4%6.5%7.5%51.0%
PassLLM13.5%28.9%1.2%0.0%39.1%
bopscrk10.3%3.9%0.0%5.6%19.2%
zh_minimalCUPP38.3%4.9%0.0%0.0%41.7%
PII2PW47.4%36.5%8.7%0.0%70.3%
PassLLM23.8%36.4%1.1%0.0%50.1%
bopscrk38.9%25.3%0.0%0.0%61.3%
zh_mediumCUPP30.4%5.2%0.0%0.2%34.3%
PII2PW40.2%27.6%9.4%8.9%61.0%
PassLLM12.9%28.5%0.8%0.0%33.9%
bopscrk35.1%3.7%0.0%5.6%38.1%
Figure 2.PII embedding rate by category on the zh_full profile.

4.Academic Comparison: Success Rate @ N

The primary metric in the targeted password guessing literature. Given a PII–password paired dataset, what fraction of target passwords appears within the first N guesses? PII2PW, CUPP and bopscrk are evaluated on 200 synthetic records modelled after real Chinese password patterns (SR@N shown where a candidate ordering was captured); the remaining baselines are reported numbers from their respective papers on real leaked corpora — see fair-comparison caveat in §6.

Table 3.Success rate at guess budget N. Asterisks mark per-column maxima. In measured rows an em dash means no SR@N was captured: CUPP emits no likelihood ordering, so only its coverage is meaningful, and PassLLM needs a GPU pass not run here; bopscrk's SR@N reflects its raw output order, not a likelihood ranking. Measured and baseline rows come from different corpora and are not directly comparable — see §6. In baseline rows an em dash means the value was not reported in the source.
MethodVenueApproachSR@10SR@100SR@1000SR@10⁴
CUPPmeasuredRule-based
PII2PWmeasuredRule-based (PII)11.0%46.5%76.5%97.5%
PassLLMmeasuredLLM (7B) + LoRA
bopscrkmeasuredRule-based3.5%6.5%11.5%11.5%
TarGuess-IIICCS 2016PII-tagged PCFG4.6%19.7%45.4%
Personal-PCFGINFOCOM 2016PCFG + PII tags12.8%29.5%
RFGuess-PIIUSENIX 2023Random forest7.3%24.1%48.7%
PointerGuessUSENIX 2024Seq2Seq + pointer8.2%25.2%
PassLLM-IUSENIX 2025LLM (7B) + LoRA9.8%31.6%52.3%
RankGuess-PIIS&P 2025RL + ranking27.8%50.1%
Figure 3.Success rate at guess budget N, measured on synthetic records. The baseline curves come from their papers' own corpora and are drawn for context, not as a like-for-like comparison.

5.Guess-Number and Distribution Statistics

For passwords that were ultimately found, at what rank in the generated list did they appear? Lower ranks indicate better priority ordering.

Table 4.Hit-rank statistics on 200 synthetic targets, over the ranks each target's own generated dictionary produced it at.
ToolFoundMissedCoverageMinMedianMeanMax
CUPP4515522.5%
PII2PW195597.5%13181,2709,749
bopscrk2317711.5%168145737

The two tools also produce nearly disjoint candidate sets, suggesting they could be productively combined. Length distributions differ markedly: CUPP concentrates 82% of its output in the 9–12 character range, while PII2PW spreads across 1–24 characters in line with observed Chinese password length distributions.

Table 5.Length distribution on the zh_full profile.
LengthCUPPCUPP %PII2PWPII2PW %PassLLMPassLLM %bopscrkbopscrk %
1–64862%3,05318%00%640%
7–84,52616%2,75216%904%5923%
9–1223,63583%6,12336%53125%5,74729%
13–1600%3,50721%1,51271%13,27267%
17–2400%1,5179%00%00%
25+00%1481%00%00%
Figure 4a.Candidate-set overlap (zh_full).
Figure 4b.Length distribution (zh_full, % of output).

6.Discussion

Strengths.

PII2PW reaches SR@100 = 46.5% and SR@10⁴ = 97.5% on our set, up from 1.0% and 84.0% before the generation order was reworked. The gain at small N came entirely from ranking: the previous static priority list emitted roughly 2,000 single-component variants before it reached name+birthdate, the most common Chinese password shape, so correct guesses landed near rank 2,800. Its 51.0% PII embedding rate on zh_full — versus CUPP’s 33.2% — confirms more targeted candidate generation. PII2PW is roughly 6× faster than CUPP with zero training data, zero GPU, and no dependency beyond a pip install.

Limitations.

The median hit-rank is 318: PII2PW still places most correct passwords in the hundreds, not the first dozen, and SR@10 = 11.0% is where a rule engine shows its ceiling. Its ordering is a hand-authored stage list, not a learned likelihood — it cannot rank two candidates within a stage, which is precisely what a trained model does. Replacing the static priority with a learned scorer over the same candidate set is the natural next step, and needs far less data than training a generator: only (profile, candidates, which one was correct). Separately, the built-in common-passwords dataset is no longer an independent test set — PII2PW now ships a generic weak-password fallback that overlaps it by construction, which is why its hit rate there jumped from 8.2% to 71.3%. Judge hit rate on rockyou instead.

Fair-comparison caveat.

The academic baselines (TarGuess, PassLLM, etc.) were evaluated on real leaked PII–password datasets — typically 12306, Dodonew — with 10⁵-plus records. Our evaluation uses 200 synthetic records modelled after published Chinese password patterns. The comparison is therefore directionally informative but not strictly equivalent.

Positioning.

PII2PW occupies a distinct niche: to our knowledge it is the only actively maintained, rule-based, Chinese-localised password profiling tool that requires neither training data nor GPU. That makes it a pragmatic option for penetration testers who cannot deploy machine-learning infrastructure — but its numbers here and the published baselines are measured on different corpora and should not be read as a ranking.

7.References

#ReferenceVenue
[1]Wang et al., Targeted Online Password Guessing: An Underestimated Threat.ACM CCS 2016
[2]Li, Wang & Sun, A Study of Personal Information in Human-chosen Passwords and Its Security Implications.IEEE INFOCOM 2016
[3]Li, Han & Xu, A Large-Scale Empirical Analysis of Chinese Web Passwords.USENIX Sec. 2014
[4]Wang & Zou, Password Guessing Using Random Forest.USENIX Sec. 2023
[5]Xiu & Wang, PointerGuess: Targeted Password Guessing Using Pointer Mechanism.USENIX Sec. 2024
[6]Zou & Wang, Password Guessing Using Large Language Models.USENIX Sec. 2025
[7]Yang & Wang, RankGuess: Password Guessing Using Adversarial Ranking.IEEE S&P 2025