PCS Methodology & Citations — A SIOP-Aligned Construct-Validity Framework

A pre-draft outline of the methodology and citation framework underpinning the Personal Competency Score (PCS). This document accompanies the PCS Validation Whitepaper (v1.0). Full sections are in active development; this page shows the outline structure with section-level honesty markers so readers can see what is documented, what is pending, and what is deferred.

Author: Talentopian Research.
Version: v0 outline (pre-draft scaffold).
Status: Pre-draft scaffold — not peer-reviewed; not finalized for citation. Full sections fill in alongside the v1.0 PCS Validation whitepaper. A v1 publish-ready draft is targeted once the framework layer described in §3 is consolidated.
Conforms to: SIOP Principles for the Validation and Use of Personnel Selection Procedures (5th ed., 2018) — the industry standard for construct-validity evidence at the technical-report tier. SIOP Principles do not require peer review for technical-report legitimacy.
License: © 2026 Talentopian. Self-archival permitted with attribution.

Abstract (target 200 words — outline form)

Problem. Traditional career assessments (MBTI, Holland, Big Five) rely on self-report, which is gameable and culturally narrow.

Approach. Talentopian's Personal Competency Score (PCS) measures behavior across 48 parameters arrayed in 8 categories through 48 production browser games (no headset; sub-15-min sessions), with localization across four languages (en-CA, ko-KR, fr-CA, es-ES).

Data. An early pilot cohort (reported in the v1.0 PCS Validation whitepaper), with a larger cross-locale cohort targeted from the Free Tier launch; the validation roadmap waves are pre-registered in the PCS Validation whitepaper §4.5.

Findings (preliminary). Preliminary internal-structure analysis on an early pilot cohort is underway; findings will inform framework consolidation and formal validation through academic partnership. Construct-validity coefficients per SIOP Principles (test-retest, convergent, criterion, discriminant) are not yet collected and are pre-registered for a future validation wave.

Contribution. First open-source-aligned, multi-locale game-based career assessment with SIOP Principles-conformant technical-report documentation, honest-by-construction overclaim guards, and a pre-registered cross-method validation roadmap.

Table of Contents

1. Introduction

Status: outline only — full section pending Week-2 draft.

Career mismatch is a global economic loss problem (cf. Strada-Gallup 2018; OECD Education at a Glance 2024). The Introduction will frame why behavioral game data improves on self-report career inventories (cf. Khanna et al., 2024 in IO Psychology; Landers et al., 2022) and will state the central hypothesis: behavioral game performance, when measured across the 48-parameter / 8-category PCS framework, predicts career fit better than short-form self-report inventories at the parameter and at the category-aggregate level.

2. Theoretical Foundation

Status: outline only — full section pending.

The theoretical anchors are:

• Holland's RIASEC (1997) — six vocational personality types.
• Social Cognitive Career Theory (Lent, Brown, & Hackett, 1994) — self-efficacy and outcome expectations.
• Big Five (Costa & McCrae, 1992) — Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism.
• O*NET occupation x skill mapping (Peterson et al., 2001).
• Korean Career Resources Scale (KCRS) — KCI-listed, Korean-context anchor.
• Evidence-Centered Design (Mislevy, Steinberg, & Almond, 2003) — the conceptual frame for game-based assessment design.
• Stealth Assessment (Shute & Ventura, 2013; Shute et al., 2016) — the embeddability principle.
• Leutner et al. (2023) — r = 0.50 convergent and r = 0.68 test-retest benchmarks for game-based cognitive ability (target benchmark).
• Maggiori, Rossier, & Savickas (2017) — Career Adapt-Abilities Scale Short Form.
• Whiston, Li, Goodrich Mitts, & Wright (2017) — counselor-mediated outcomes meta-analysis (flagship cite for the counselor distribution route).

3. PCS Construct: 48 Parameters / 8 Categories

Status: canonical structure confirmed; full section text pending.

4. Methodology

Status: cross-references the v1.0 PCS Validation whitepaper §3; SIOP-aligned full section text pending.

Per the v1.0 PCS Validation whitepaper §3: short browser-based games (roughly one to six minutes each) emit per-trial behavioral events; the scoring pipeline reduces these to parameter-level 0–100 scores via a per-game contribution map. Scoring is scale-aware — games that report on different internal raw-score ranges are calibrated per game before any inter-game aggregation, so no single game's native scale dominates. Career match runs over an O*NET-anchored occupational taxonomy using a similarity comparison over the PCS vector, with a WEF Future of Jobs industry overlay. The SIOP-aligned text in this section will detail evidence-source ladders (operational test, performance-based, criterion-related) and explicitly identify the SIOP Principles evidence type each PCS data path satisfies.

4.1 Parameter-to-career bridge

A parameter-to-career matching layer maps assessed parameters onto occupations, reconciling the several vocabularies that describe competencies across the assessment framework, the matching layer, and the user-facing competency bars. A subset of bridge skills aggregates into a compact set of soft-skill dimensions used for matching, while more granular cognitive skills collapse into higher-order problem-solving and critical-thinking dimensions.

5. Construct Validity Tests (pre-registered)

Status: hypothesis matrix pre-registered in the v1.0 PCS Validation whitepaper §4.5; confirmatory coefficients pending data collection.

5.1 Convergent

• PCS Analytical Skills (Strategic) r ≥ 0.55 with Wonderlic Cognitive Ability Test (online surrogate).
• PCS Communication Skills r ≥ 0.50 with Big Five Openness.
• PCS Ethical & Moral Reasoning r ≥ 0.40 with Big Five Agreeableness.

5.2 Discriminant

• PCS Cognitive Skills (General, spatial subset) r ≤ 0.30 with Big Five Neuroticism (should not correlate).

5.3 Predictive (longitudinal, Year 2 add)

• Outcome: actual career entry at 6, 12, and 24 months post-assessment via an opt-in placement-intent field.
• Hypothesis: top-3 PCS recommendation hit rate materially exceeds the career-mismatch base rate; specific thresholds are pre-registered for the confirmatory wave.

5.4 Occupational-anchor validity (bridge-layer evidence)

Beyond the parameter-level construct-validity hypotheses (5.1–5.3), the parameter-to-career bridge (§4.1) supplies a second-tier validity surface: the bridge's per-pair relevance scores can be analyzed for internal consistency against established O*NET knowledge/skill/ability (KSA) importance ratings. Pre-registered hypothesis: the bridge relevance score for a (career, bridge-skill) pair correlates r ≥ 0.40 with the O*NET KSA importance for the same career and the cross-walked KSA element. This is a within-bridge validity check; it does not substitute for the convergent and criterion validation studies in 5.1–5.3, but it provides early evidence that the matching layer is occupationally coherent. This within-bridge validity analysis is planned for a future validity study.

5.5 Category-level aggregation

The bridge is also aggregated to a compact set of soft-skill dimensions used for matching, each carrying an averaged relevance over its constituent bridge parameters. Where a dimension's constituent parameters have no coverage for a given career, the value is preserved as absent (null) rather than imputed as zero, so “no claim” is never silently read as “low score”.

An illustrative mapping of granular skills to higher-order dimensions:

• problem-solving ← problem-solving + decision-making + pattern-recognition + spatial-reasoning
• critical-thinking ← critical-thinking + analytical + logical + attention + memory
• adaptability ← adaptability + time-management
• Remaining dimensions (communication, leadership, teamwork, emotional-intelligence, creativity) map directly from their bridge-skill counterparts.

Per-dimension career coverage is variable and is reported honestly rather than collapsed: coverage is substantial for dimensions such as leadership and emotional-intelligence, and sparse for creativity. Face-valid sample outputs are sensible — e.g., a chief-executive career leads on problem-solving, communication, and leadership; a registered-nurse career leads on communication and emotional-intelligence. Per-dimension correlation reporting includes sub-sample size so that low-coverage dimensions are weighted appropriately rather than folded into a single average.

6. Fairness Audit (SIOP 2018 §13)

Status: 4/5ths-rule plan documented; full section pending.

Adverse-impact ratios will be reported per game and per parameter across gender x age cohort, applying the EEOC (1978) 4/5ths rule. Cohort-size threshold for publishing dim-level fairness data is N ≥ 100 per cohort; the v0 outline does not report fairness numbers because the threshold is not yet reached. Any parameter whose lowest-scoring protected-group ratio falls below 4/5ths of the highest-scoring cohort will be flagged for re-weighting prior to confirmatory use.

7. Multi-Locale Validation

Status: translation protocol established; locale-cohort data pending sub-threshold.

• Translation: native + back-translation per Brislin (1970).
• Korean Career Resources Scale (KCRS) cross-walk: KCI-listed, peer-reviewed, used as the KO-context measurement anchor.
• Cohort-size threshold for publishing dim-level locale data: N ≥ 100 per locale. The platform is currently sub-threshold across all locales; we will not publish locale-comparative claims at v1 of this whitepaper.

8. Limitations & Future Work

Status: outline — limitations enumerated honestly; formal validation pending academic partnership.

8.1 Cohort + sample limitations

• Cohort size. The early pilot cohort (formal validation pending) is the binding limit on coefficient inference (the v1.0 PCS Validation whitepaper §3 + §5). Preliminary internal-structure analysis is treated as exploratory only, with any near-perfect correlations read cautiously given the small sample.
• Self-selection bias. Free Tier users are not a probability sample; sponsored-seat cohorts provide a partial control. A larger locale-specific cohort, accrued through counseling-community engagement, is the anticipated path to closing this gap at the locale level.
• Predictive validity. Requires 12–24-month longitudinal follow-up; opt-in placement-intent capture is in place; the first measurable wave depends on reaching a sufficient cohort size.

8.2 Bridge-coverage limitations

• The parameter-to-career bridge covers only a subset of the 48 framework parameters. Per §4.1, the bridge covers the bridge-skill subset that maps to the compact soft-skill dimensions used for matching; the four PHYSICAL parameters plus the personality and values parameters are bridge-inert by O*NET design. Career-match output today aggregates only over the bridged skills.
• Per-dimension coverage variance. Per §5.5, per-career coverage varies widely across dimensions (substantial for some, sparse for creativity). Per-dimension correlation reporting includes sub-sample size to prevent collapse-to-average bias.
• Weighting provenance. The matching weights are not yet fully derived from the bridge; grounding those weights in the bridge is an anticipated follow-up so that every recommendation traces to a documented competency source.

8.3 PHYSICAL adoption sparsity (bridge-inert by design)

• The PHYSICAL score channel is functional but sparsely populated. Few users have accrued PHYSICAL parameter scores to date; the measurement channel is end-to-end inspectable from game emission through scoring, but adoption at scale is not yet present (these plays accrue through reward games unlocked later in the free-tier journey).
• PHYSICAL stays bridge-inert by O*NET design. PHYSICAL parameters do not enter the soft-skill matching dimensions; they support the anti-faking measurement thesis but do not extend the career-match hypotheses. The four PHYSICAL parameters are retained by design (they power reward-game and camera-resistant measurement, not career match).

8.4 Career-resilience evidence (planned)

• Labor-market resilience signals. A separate line of analysis examines which occupations show a higher share of tasks resistant to automation (human judgment, in-person interaction, licensed and physical work). This external evidence is intended to complement the assessment's construct-validity work; hypothesis-level results are deferred until the underlying occupational cross-walk is complete.

8.5 Dissemination and community engagement

• Academic dissemination. The methodology is being prepared for submission to methodological and practitioner career-development venues, alongside the self-published research reports. A future peer-reviewed, IRB-gated study is planned through academic partnership.
• Korean career-counseling community. The framework is designed to be relevant to Korean career-counseling practice, including a Korean-context measurement anchor (see §7). We hope to engage the Korean career-counseling community as a partner for locale-specific cohort accrual and cross-validation.

9. Open Science Commitment

Status: artifact intent documented; companion materials not yet public.

• An openly licensed excerpt of the PCS scoring framework to be published as a companion reference.
• De-identified cohort data via a dedicated research-data access page (FERPA / PIPA / LGPD compliant).
• Co-research invitations to Korean counseling-psychology PhD programs.

Matching weights are being moved toward a provenance-grounded footing so that each recommendation can be traced to its per-user competency scores and the documented competency-to-career bridge, closing the auditability loop the SIOP Principles (2018) technical-report standard expects. This grounds the weights in documented sources; the scoring formula itself remains private by design, and an upstream coherence check for anti-faking runs ahead of matching regardless.

10. References (sample — APA 7th edition)

Status: sample list; full APA reference list pending.

• Brislin, R. W. (1970). Back-translation for cross-cultural research. Journal of Cross-Cultural Psychology, 1(3), 185–216.
• Costa, P. T., & McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO PI-R). Psychological Assessment Resources.
• EEOC. (1978). Uniform Guidelines on Employee Selection Procedures (4/5ths rule). 29 CFR Part 1607.
• Holland, J. L. (1997). Making vocational choices: A theory of vocational personalities and work environments (3rd ed.). Psychological Assessment Resources.
• Khanna, V., et al. (2024). Game-based behavioral assessment for personnel selection — meta-analysis. Industrial & Organizational Psychology.
• Landers, R. N., et al. (2022). The validity of gamified assessments — IO Psychology review.
• Lent, R. W., Brown, S. D., & Hackett, G. (1994). Toward a unifying social cognitive theory of career and academic interest, choice, and performance. Journal of Vocational Behavior, 45(1), 79–122.
• Leutner, F., Codreanu, S.-C., Brink, S., & Bitsakis, T. (2023). Game-based assessment of cognitive ability for recruitment. Frontiers in Psychology, 13:942662.
• Maggiori, C., Rossier, J., & Savickas, M. L. (2017). Career Adapt-Abilities Scale Short Form (CAAS-SF). Journal of Career Assessment, 25(2).
• Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement, 1(1), 3–62.
• Peterson, N. G., et al. (2001). Understanding work using the Occupational Information Network (O*NET). Personnel Psychology, 54(2), 451–492.
• SIOP. (2018). Principles for the Validation and Use of Personnel Selection Procedures (5th ed.).
• Shute, V. J., & Ventura, M. (2013). Stealth Assessment: Measuring and Supporting Learning in Video Games. MIT Press.
• Whiston, S. C., Li, Y., Goodrich Mitts, N., & Wright, L. (2017). Effectiveness of career choice interventions: A meta-analytic replication and extension. Journal of Vocational Behavior, 100, 175–184.

11. Appendices

Status: outline only — appendix content tracked as separate artifacts.

• A. PCS parameter → game mapping registry — the open scoring-framework specification for the 48-parameter / 8-category mapping.
• B. Normalization algorithm — the per-game calibration that satisfies the scale-aware property in §4.
• C. AI-resilience formula derivation — canonical (1 - automatability) · 0.6 + human_complement · 0.4.
• D. Fairness audit method — adverse-impact analysis approach (see the AI-Era Career Displacement whitepaper).
• E. Cohort demographics + locale × dimension × N table — surfaced when locale cohorts reach the N ≥ 100 threshold.

12. Companion artifacts

This is a pre-draft outline of the PCS methodology and citations whitepaper. It documents the canonical framework (48 parameters / 8 categories, with PHYSICAL as the eighth category), the parameter-to-career bridge and category-level aggregation used for matching, and the pre-registered construct-validity roadmap. Full remaining sections fill in alongside the v1.0 PCS Validation whitepaper (LIVE) and the peer-reviewed manuscript in preparation. — Every numeric claim on this page is a pointer to the companion source; cite the source artifact, not this outline. © 2026 Talentopian.