PCS Methodology & Citations — A SIOP-Aligned Construct-Validity Framework

A pre-draft outline of the methodology and citation framework underpinning the Personal Competency Score (PCS). This document accompanies R-01 v1.0 — PCS Validation Whitepaper and the R-08 AERA 2027 paper draft. Full sections are in active development; this page shows the outline structure with section-level honesty markers so readers can see what is documented, what is pending, and what is deferred.

Author: Jae Yong Yune (founder, Talentopian) with contributions from the Talentopian research and engineering teams.
Version: v0 outline (drafted 2026-06-20; refreshed 2026-06-25 for D-15 v0.4 framework update).
Status: Pre-draft scaffold — not peer-reviewed; not finalized for citation. Full sections fill in alongside R-01 v1.0 and R-08 AERA 2027 development. A v1 publish-ready draft is targeted for completion ~8 weeks after sajangnim sign-off on §3 framework canonical layer.
Conforms to: SIOP Principles for the Validation and Use of Personnel Selection Procedures (5th ed., 2018) — the industry standard for construct-validity evidence at the technical-report tier. SIOP Principles do not require peer review for technical-report legitimacy.
License: © 2026 Talentopian. Self-archival permitted with attribution.

Abstract (target 200 words — outline form)

Problem. Traditional career assessments (MBTI, Holland, Big Five) rely on self-report, which is gameable and culturally narrow.

Approach. Talentopian's Personal Competency Score (PCS) measures behavior across 48 parameters arrayed in 8 categories through 48 production browser games (no headset; sub-15-min sessions), with localization across four languages (en-CA, ko-KR, fr-CA, es-ES).

Data. Pilot N=21 backfilled users (R-01 v1.0); target N≥500 cohort by Q3 2026 from cross-locale Free Tier launch; validation roadmap waves pre-registered in R-01 §4.5 and elaborated in R-08 AERA 2027 paper Section 6.

Findings (preliminary). R-01 v1.0 reports mean |r| = 0.45 across 86 pairwise-complete correlations (n ≥ 10 per pair) with explicit two-readings disclosure for r = 1.0 clusters; construct-validity coefficients per SIOP Principles (test-retest, convergent, criterion, discriminant) are not yet collected and are pre-registered for v2.

Contribution. First open-source-aligned, multi-locale game-based career assessment with SIOP Principles-conformant technical-report documentation, honest-by-construction overclaim guards, and a pre-registered cross-method validation roadmap.

Table of Contents

1. Introduction

Status: outline only — full section pending Week-2 draft per SIOP_WHITEPAPER_OUTLINE.md schedule.

Career mismatch is a global economic loss problem (cf. Strada-Gallup 2018; OECD Education at a Glance 2024). The Introduction will frame why behavioral game data improves on self-report career inventories (cf. Khanna et al., 2024 in IO Psychology; Landers et al., 2022) and will state the central hypothesis: behavioral game performance, when measured across the 48-parameter / 8-category PCS framework, predicts career fit better than short-form self-report inventories at the parameter and at the category-aggregate level.

2. Theoretical Foundation

Status: outline only — full section pending.

The theoretical anchors are:

• Holland's RIASEC (1997) — six vocational personality types.
• Social Cognitive Career Theory (Lent, Brown, & Hackett, 1994) — self-efficacy and outcome expectations.
• Big Five (Costa & McCrae, 1992) — Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism.
• O*NET occupation x skill mapping (Peterson et al., 2001).
• Korean Career Resources Scale (KCRS) — KCI-listed, Korean-context anchor.
• Evidence-Centered Design (Mislevy, Steinberg, & Almond, 2003) — the conceptual frame for game-based assessment design.
• Stealth Assessment (Shute & Ventura, 2013; Shute et al., 2016) — the embeddability principle.
• Leutner et al. (2023) — r = 0.50 convergent and r = 0.68 test-retest benchmarks for game-based cognitive ability (target benchmark).
• Maggiori, Rossier, & Savickas (2017) — Career Adapt-Abilities Scale Short Form.
• Whiston, Li, Goodrich Mitts, & Wright (2017) — counselor-mediated outcomes meta-analysis (flagship cite for the counselor distribution route).

3. PCS Construct: 48 Parameters / 8 Categories

Status: canonical structure confirmed via D-15 v0.4 (2026-06-23); full section text pending.

4. Methodology

Status: cross-references R-01 v1.0 §3 (LIVE); D-15 param-career bridge LIVE (C7 commit 5886720a); SIOP-aligned full section text pending.

Per R-01 v1.0 §3: 48 production games (1–6 minutes each) emit per-trial events; the scoring pipeline reduces these to parameter-level 0–100 scores via a per-game contribution map (single source of truth). Scoring is scale-agnostic — three internal raw-score ranges (normalized 0–100, raw 0–1000, raw 0–5000) are calibrated per game before any inter-game aggregation. Career match runs through V2 Taxonomy 1,041 occupations with cosine-similarity over the PCS vector and a WEF Future-of-Jobs-2025 industry overlay. The SIOP-aligned text in this section will detail evidence-source ladders (operational test, performance-based, criterion-related) and explicitly identify the SIOP Principles evidence type each PCS data path satisfies.

4.1 Operational param-to-career bridge (D-15, LIVE)

The param-to-career matching layer is materialized as a BigQuery table, reference_data.pcs_param_career_bridge, containing 11,847 rows = 1,022 careers × 16 bridge skills with relevance scores per row (C7 commit 5886720a, 2026-06-25). The bridge reconciles four internal vocabularies that previously fragmented matching analytics: (a) the PARAM canonical framework (snake_case, 48 params / 8 categories per D-15 v0.4), (b) the pcs_bridge_skills taxonomy (hyphen-case, 16 skills), (c) the Gateway runtime SoftSkillProfile interface (camelCase, 8 soft skills consumed by ai-matching.service.ts), and (d) the dashboard competency bars vocabulary. All 8 SoftSkillProfile dimensions are present in the 16 bridge skills; the remaining 8 bridge skills are granular cognitive skills that collapse into problemSolving and criticalThinking in the SoftSkillProfile aggregation.

5. Construct Validity Tests (pre-registered)

Status: hypothesis matrix pre-registered in R-01 v1.0 §4.5 + R-08 AERA paper Section 6; v2 coefficients pending data collection.

5.1 Convergent

• PCS Analytical Skills (Strategic) r ≥ 0.55 with Wonderlic Cognitive Ability Test (online surrogate).
• PCS Communication Skills r ≥ 0.50 with Big Five Openness.
• PCS Ethical & Moral Reasoning r ≥ 0.40 with Big Five Agreeableness.

5.2 Discriminant

• PCS Cognitive Skills (General, spatial subset) r ≤ 0.30 with Big Five Neuroticism (should not correlate).

5.3 Predictive (longitudinal, Year 2 add)

• Outcome: actual career entry at 6, 12, and 24 months post-assessment via opt-in placement_intent Firestore field (W-08 infrastructure ready).
• Hypothesis: top-3 PCS recommendation hit rate ≥ 35% versus ~12% career-mismatch baseline.

5.4 Occupational-anchor validity (bridge-layer evidence)

Beyond the parameter-level construct-validity hypotheses (5.1–5.3), the D-15 param-career bridge (LIVE, §4.1) supplies a second-tier validity surface: the bridge's per-row relevance scores can be analyzed for internal consistency against established O*NET KSA importance ratings. Pre-registered hypothesis: across the 11,847 rows, the bridge relevance score for a (career, bridge-skill) pair correlates r ≥ 0.40 with the O*NET KSA importance for the same career and the cross-walked KSA element. This is a within-bridge validity check; it does not substitute for the convergent and criterion validation studies in 5.1–5.3, but it is computable from existing artifacts and provides early evidence that the matching layer is occupationally coherent. C7 owns the analysis (R-03 lane).

5.5 Operational SoftSkillProfile aggregation (8-dim view, LIVE)

The same D-15 16-param bridge is now also aggregated to the production-runtime SoftSkillProfile(8) layer via a second view, reference_data.pcs_softskill_8dim_view (LIVE, C7 commit bb21099a, 2026-06-25). The view contains 1,022 careers × 8 dimensions = 8,176 rows, each row carrying AVG(relevance_score) over the constituent bridge params for that dimension. NULLs are preserved where a dimension's bridge params have zero coverage for a career — per the C7 commit message, "present-only → honest NULLs, not fake 0".

The 16-to-8 mapping (verbatim from C7 commit):

• problemSolving ← problem-solving + decision-making + pattern-recognition + spatial-reasoning
• criticalThinking ← critical-thinking + analytical + logical + attention + memory
• adaptability ← adaptability + time-management
• Remaining five dims (communication, leadership, teamwork, emotionalIntelligence, creativity) map 1:1 from their bridge-skill counterparts.

Per-dim career coverage is variable and is reported honestly rather than collapsed: leadership 549 / 1,022 (53.7 %), emotional-intelligence 546 / 1,022 (53.4 %), creativity 148 / 1,022 (14.5 %). Face-valid sample outputs from C7: a CEO career leads on problemSolving + communication + leadership; a Registered Nurse leads on communication + emotionalIntelligence. Consumer-side note (LIVE-vs-runtime gap): the production code path ai-matching.service.ts is NOT yet rewired to read from this view; the existing hardcoded SoftSkillProfile weights remain authoritative at runtime. R-04 and the bridge-validity analyses below analyze this BQ view directly, NOT the production code path. The "runtime SoftSkillProfile reads from BQ view" rewire is a tracked C1 + C7 follow-up.

5.6 Cross-validity with R-04 anti-copilot signals (planned)

Combining the 8-dim view (§5.5) with C7's postings_anti_copilot_signals (LIVE, commit d60f263b, 123,842 postings, 18 % at score ≥ 2) yields the well-powered workhorse hypothesis of R-04 Anti-Copilot Career Resilience v2: for each of the eight SoftSkillProfile dimensions, the per-career dim mean correlates with per-career anti-copilot rate at Pearson r ≥ 0.30, well-powered at N=1,022. The pre-registered priority dims are problemSolving (predicted increase per Eloundou et al., 2023) and emotionalIntelligence (predicted increase per Whiston et al., 2017 counselor-mediated outcomes literature). Per-dim coverage variance (especially creativity at 14.5 %) is reported alongside each per-dim r so readers can weight the result by sub-sample N rather than treating the analysis as a single average.

6. Fairness Audit (SIOP 2018 §13)

Status: 4/5ths-rule plan owned by C7 in R-03 lane; full section pending.

Adverse-impact ratios will be reported per game and per parameter across gender x age cohort, applying the EEOC (1978) 4/5ths rule. Cohort-size threshold for publishing dim-level fairness data is N ≥ 100 per cohort; the v0 outline does not report fairness numbers because the threshold is not yet reached. Any parameter whose lowest-scoring protected-group ratio falls below 4/5ths of the highest-scoring cohort will be flagged for re-weighting prior to confirmatory use.

7. Multi-Locale Validation

Status: translation protocol established; locale-cohort data pending sub-threshold.

• Translation: native + back-translation per Brislin (1970).
• Korean Career Resources Scale (KCRS) cross-walk: KCI-listed, peer-reviewed, used as the KO-context measurement anchor.
• Cohort-size threshold for publishing dim-level locale data: N ≥ 100 per locale. The platform is currently sub-threshold across all locales; we will not publish locale-comparative claims at v1 of this whitepaper.

8. Limitations & Future Work

Status: enriched 2026-06-25 (Turn 22) with D-15 16-of-48 bridge gap, R-04 cross-link, R-08 AERA Section 6 publication-path, KCA Aug 1 pilot trigger, C7 title_soc_bridge dependency, PHYSICAL adoption sparsity.

8.1 Cohort + sample limitations (carried)

• Cohort size. N=21 PCS-computed pilot is acknowledged as the binding limit on coefficient inference (R-01 v1.0 §3 + §5; AERA paper draft v3 §5 honestly marks Reading A vs Reading B for the r = 1.0 clusters at this n).
• Self-selection bias. Free Tier users are not a probability sample; sponsored-seat cohorts (Q4 2026) provide a partial control. The KCA Aug 1 partnership window (R-10 calendar T1-3) is the proposed Wave 1 trigger for N ≥ 500 Korean cohort accrual that closes this gap at the locale level.
• Predictive validity. Requires 12–24-month longitudinal follow-up; placement_intent opt-in infrastructure is in place; first measurable wave depends on Wave 1 N ≥ 500 cohort start.

8.2 Bridge-coverage limitations (D-15 16-of-48 gap)

• Param-to-career bridge covers 16 of 48 framework parameters. Per §4.1, the operational bridge (pcs_param_career_bridge) covers the 16 bridge-skill subset that maps to SoftSkillProfile(8); the four PHYSICAL params + six PERSONALITY params + two VALUES params are bridge-inert by O*NET design. Career-match output today aggregates only over the 16 bridged skills.
• SoftSkillProfile(8) per-dim coverage variance. Per §5.5, per-career coverage ranges from creativity 14.5 % (148 / 1,022) to leadership 53.7 % (549 / 1,022). Per-dim correlation reporting includes sub-sample N to prevent collapse-to-average bias.
• LIVE-vs-runtime gap. Per §5.5, the production code path ai-matching.service.ts is NOT yet rewired to read from pcs_softskill_8dim_view; runtime weights remain hardcoded. Tracked as a C1 + C7 follow-up; analyses cited here analyze the BQ view directly.

8.3 PHYSICAL adoption sparsity (NEW v4 — bridge-inert by design)

• PHYSICAL adoption-score channel LIVE but sparse. The Turn 22 C7 commit 3c9b5200 ships qa_data.physical_params_snapshot with 15 user-param-score rows across 4 PHYSICAL params (auditoryProcessing 9 users; environmentalAwareness 4 users; handEyeCoordination 1 user; physicalStamina 1 user). The score channel is end-to-end inspectable from game emission → A-05 dual-write → BQ — the W-22 anti-faking moat's measurement-channel pillar is auditable. Adoption-at-scale is NOT yet present (reward-game plays accrue post Free-Tier N=15 unlock).
• PHYSICAL stays bridge-inert by O*NET design. PHYSICAL params do NOT enter pcs_softskill_8dim_view; they support the W-22 moat measurement-channel pillar (R-04 v4 §5.7) but do NOT extend career-match hypotheses (H1 / H2 / H4 / H6). DO-NOT-deprecate the four PHYSICAL params (long-standing directive per memory physical_params_reward_strategy.md): they power the reward-game / camera-bypass moat, NOT career match.

8.4 Anti-copilot evidence cross-link (NEW)

• R-04 anti-copilot research stream LIVE evidence cross-link. The R-04 v4 draft (`R04_anti_copilot_v4.md`, Turn 22 commit dd3bb5e8) cites C7's r04_anti_copilot_aggregated view (commit a32e6a28): 22,473 / 123,842 = 18.15 % of US postings show ≥ 2 anti-copilot signals; signal-weight matrix human_judgment 24 % > in_person 21 % > licensed 16 % > physical 14.5 % > safety 4.6 %. This evidence supports the W-22 moat thesis externally; combined with W-21 Coherence Triangulation (LIVE) and PHYSICAL adoption channel (LIVE, sparse), the moat composes three independently auditable pillars.
• Title-vs-SOC aggregation gap (R-04 H1/H2/H4/H6 blocker). R-04 hypothesis-testing analyses are pending C7's title_soc_bridge follow-up join. Until that lands, R-04 v4 reports descriptive evidence (§3.4 / §3.5 / §3.7) honestly and defers hypothesis result reporting to v5+.

8.5 Cross-venue dissemination (R-08 AERA + Cannexus 28 + NCDA 2027) and KCA Aug 1 trigger

• Academic venue. R-08 AERA 2027 paper draft v3 (commit 2f1e0809, deadline 2026-07-24) Section 6 publication-path commitment names Cannexus 28 (Jan 2028, submission ~April 2027) and NCDA 2027 Anaheim (June 29 — July 1) alongside the AERA Toronto place-based submission. AERA is the methodological audience; Cannexus + NCDA are the practitioner audience.
• KCA Aug 1 partnership. The 한국상담학회 (KCA, 24K members) endorsement decision window 2026-08-01 (R-10 calendar T1-3) is the proposed Wave 1 trigger for cohort N ≥ 500. KCA Deck v4 finalize (Turn 20 commit 45bf81b5) Appendix E.5 cites the Turn 20 D-15 SOC reconciliation 100 % join coverage + ALL 4 reward games LIVE as production-readiness evidence sajangnim can cite in the meeting.

9. Open Science Commitment

Status: artifact intent documented; companion repo not yet public.

• MIT-license PCS scoring engine excerpt to be published in a companion repository.
• De-identified cohort data via a dedicated /research/data access page (FERPA / PIPA / LGPD compliant).
• Co-research invitations to KR counseling-psychology PhD programs (R-05 MoU lane).

9.5 ai-matching Integration Roadmap (D-15 follow-up, C7 proposal carry)

Status: enriched 2026-06-25 (Turn 25) — C7 commit cf867d14 formal proposal to wire ai-matching.service.ts skillWeights to LIVE BQ provenance source.

The Methodology section (§4 + §5.5) reports the production runtime career-match path: ai-matching.service.ts consumes a SoftSkillProfile(8) per user and outputs a ranked career list from the V2 taxonomy. The LIVE-vs-runtime gap previously noted in §5.5 was: the per-career skillWeights that drive the cosine-similarity step remain hardcoded in the Gateway source, while the BQ-grounded provenance view reference_data.pcs_softskill_8dim_view (LIVE per C7 commit bb21099a, 1,022 careers × 8 dims) has been ready to consume since Turn 19.

Turn 25 carries the formal C7 proposal (commit cf867d14) to wire the consumer side. The integration design has the following elements:

9.5.1 Data source — LIVE

• Reference view: reference_data.pcs_softskill_8dim_view — 848 careers × 8 SoftSkillProfile dimensions (post-D-15 SOC reconciliation per C7 commit 4232c0d5 — 1,022 ONET codes collapse to 848 unique SOC6 codes).
• Per-dim coverage: reported honestly per §5.5 (leadership 53.7 % / EI 53.4 % / creativity 14.5 %). NULLs preserved (no fake-zero imputation).
• SOC join key: career soc6 ↔ user user_pcs_summary.soc6 (12 / 12 + 22 / 22 = 100 % join coverage per C7 Turn 20 commit 4232c0d5 SOC reconciliation).

9.5.2 Normalization plan

• Per-dim min-max normalization to 0–100 range across the 848 careers.
• NULL handling: preserved at the view level; downstream consumer treats NULL as “no claim” rather than zero. The hardcoded weights this replaces did not have a NULL state, so the consumer-side default-NULL handling is a behavioral change — flagged as a v1 implementation note for the Gateway change owner.

9.5.3 Consumer-side rewire (Gateway)

• Component: ai-matching.service.ts in the Gateway. Replace the hardcoded per-career skillWeights map with a runtime read of the normalized 8-dim view (cached per-day or per-deploy — the view is reference data, not user data).
• Provenance audit trail: after rewire, every recommendation can be traced to (a) per-user SoftSkillProfile(8) scores, (b) per-career bridge-derived weights from the 16-param O*NET bridge, (c) the explicit per-dim coverage at the per-career level — closing the auditability loop that the SIOP Principles 2018 technical-report standard expects for selection-decision provenance.
• Status (2026-06-25, Turn 25): proposal LIVE in C1 inbox via C7 commit cf867d14; Gateway change owner is C1; non-blocking for ongoing W-21 / param pipelines.

9.5.4 Honesty boundaries (§13 / N-09)

• This roadmap item is about provenance auditability, NOT about opening the algorithm code itself. R-10 (open-source PCS engine paper) remains DEFERRED per sajangnim 2026-04 transcript stance and Turn 23 RESOLVED ack (“알고리즘 비공개”). The rewire makes the WEIGHTS provenance-grounded; the SCORING FORMULA stays internal.
• Per-dim coverage variance reporting (§5.5 honest framing) must be carried forward into the consumer-side: recommendations whose top-3 careers rely on low-coverage dims (e.g., creativity at 14.5 %) should be honestly flagged in the explanation surface.
• The Coherence Triangulation Validator (W-21) gate runs upstream of ai-matching; rewiring weights does NOT bypass the anti-faking gate.

10. References (sample — APA 7th edition)

Status: sample list; full APA reference list pending.

• Brislin, R. W. (1970). Back-translation for cross-cultural research. Journal of Cross-Cultural Psychology, 1(3), 185–216.
• Costa, P. T., & McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO PI-R). Psychological Assessment Resources.
• EEOC. (1978). Uniform Guidelines on Employee Selection Procedures (4/5ths rule). 29 CFR Part 1607.
• Holland, J. L. (1997). Making vocational choices: A theory of vocational personalities and work environments (3rd ed.). Psychological Assessment Resources.
• Khanna, V., et al. (2024). Game-based behavioral assessment for personnel selection — meta-analysis. Industrial & Organizational Psychology.
• Landers, R. N., et al. (2022). The validity of gamified assessments — IO Psychology review.
• Lent, R. W., Brown, S. D., & Hackett, G. (1994). Toward a unifying social cognitive theory of career and academic interest, choice, and performance. Journal of Vocational Behavior, 45(1), 79–122.
• Leutner, F., Codreanu, S.-C., Brink, S., & Bitsakis, T. (2023). Game-based assessment of cognitive ability for recruitment. Frontiers in Psychology, 13:942662.
• Maggiori, C., Rossier, J., & Savickas, M. L. (2017). Career Adapt-Abilities Scale Short Form (CAAS-SF). Journal of Career Assessment, 25(2).
• Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement, 1(1), 3–62.
• Peterson, N. G., et al. (2001). Understanding work using the Occupational Information Network (O*NET). Personnel Psychology, 54(2), 451–492.
• SIOP. (2018). Principles for the Validation and Use of Personnel Selection Procedures (5th ed.).
• Shute, V. J., & Ventura, M. (2013). Stealth Assessment: Measuring and Supporting Learning in Video Games. MIT Press.
• Whiston, S. C., Li, Y., Goodrich Mitts, N., & Wright, L. (2017). Effectiveness of career choice interventions: A meta-analytic replication and extension. Journal of Vocational Behavior, 100, 175–184.

11. Appendices

Status: outline only — appendix content tracked as separate artifacts.

• A. PCS parameter → game mapping registry — links to shared/pcs-dim-map.js SSoT (D-15 v0.4 48-param / 8-cat).
• B. Normalization algorithm — links to shared/normalization.js (the per-game calibration that satisfies the scale-agnostic property in §4).
• C. AI-resilience formula derivation — canonical (1 - automatability) · 0.6 + human_complement · 0.4.
• D. Fairness audit BQ queries — R-03 reproducibility kit.
• E. Cohort demographics + locale x dim x N table — surfaced when locale cohorts reach N ≥ 100 threshold.

12. Companion artifacts

Outline drafted 2026-06-20 by C1; refreshed 2026-06-25 by C5 to absorb the D-15 v0.4 framework canonical (48 parameters / 8 categories with PHYSICAL added as the 8th category) and to remove the superseded Appendix-G "44 → 10 canonical" mapping. Polished 2026-06-25 by C5 (Turn 18) to add §4.1 operational param-to-career bridge (LIVE; 11,847 rows = 1,022 careers × 16 bridge skills per C7 commit 5886720a) and §5.4 occupational-anchor validity hypothesis. Further enriched 2026-06-25 by C5 (Turn 19) with §5.5 SoftSkillProfile(8) aggregation LIVE (1,022 careers × 8 dims = 8,176 rows per C7 commit bb21099a) and §5.6 cross-validity link to R-04 anti-copilot v2 (commit 9df42fff) workhorse N=1,022 Pearson r. Polished 2026-06-25 by C5 (Turn 25) with NEW §9.5 ai-matching Integration Roadmap (C7 proposal carry per commit cf867d14) detailing the LIVE-vs-runtime gap rewire from hardcoded skillWeights to BQ-grounded pcs_softskill_8dim_view provenance source. Full remaining sections fill in alongside R-01 v1.0 (LIVE) and R-08 AERA 2027 (in flight to 2026-07-24 deadline). — Every numeric claim on this page is a pointer to the underlying SSoT; cite the source artifact, not this outline. © 2026 Talentopian.