Report of the Academic Senate Smarter Balanced Study Group

September 2021 1

REPORT OF THE ACADEMIC SENATE SMARTER BALANCED STUDY GROUP

RECOMMENDATION

The Smarter Balanced Study Group recommends that the Smarter Balanced

assessment should not be used in the UC admissions process.

INTRODUCTION

In May of 2020, the Board of Regents of the University of California voted to suspend

the use of scores from the SAT/ACT standardized tests in the UC freshman application

for admission until 2024. This decision, which is part of the ongoing effort by the

university to advance educational opportunity and equity, was based on the view that

these tests are biased because they systematically and unfairly reduce the likelihood

that underrepresented and low-income high school students will be accepted to the

university. The decision was followed by a proposal from then UC President Janet

Napolitano to form a work group to determine the feasibility of creating or identifying a

replacement standardized test for UC freshman applications that is fair to the student,

useful in the admissions process, and ready to implement in the fall of 2025.

In fall of 2020, the Feasibility Study Steering Committee

(https://regents.universityofcalifornia.edu/regmeet/jan21/b2attach1.pdf) and the

Feasibility Study Work Group

(https://regents.universityofcalifornia.edu/regmeet/jan21/b2attach3.pdf) were convened

for this purpose. These groups completed their work in December 2020 with three

conclusions: (1) it is not feasible for the university to develop its own test in the specified

time frame, (2) modification of the SAT/ACT is not a viable option given long-standing

concerns about the fairness of these tests, and (3) it may be feasible to leverage an

existing test, under certain conditions, for use in UC freshman admissions. The groups

suggested exploration of the state’s Smarter Balanced (SB) assessment given that,

among other things, it is already required of all California public school students in 11

grade, it aligns with the state’s Common Core academic standards, and it is

administered free of charge.

THE SMARTER BALANCED STUDY GROUP

In April of 2021, UC President Drake asked the Academic Senate to undertake an

exploration of the Smarter Balanced assessment to determine if it can provide added

value in the UC admissions process in an equitable manner. In response, the Academic

September 2021 2

Senate created the Smarter Balanced Study Group (SBSG), composed of eight faculty

members from across the UC system with expertise in educational testing and policy,

co-chaired by Mary Gauvain, Chair of the UC Academic Senate, and Madeleine

Sorapure, Vice Chair of the Senate Board of Admissions and Relations with Schools

(BOARS), and staffed by two members of the UC President’s office (Roster attached).

SBSG Charge. The President asked the SBSG to explore the following questions using

the BOARS principles on admissions testing

(https://regents.universityofcalifornia.edu/regmeet/mar20/b4attach2.pdf).

Q1. What is the current evidence that Smarter Balanced assessment scores, either

alone or when paired with HSGPA, correlate with UC freshman admission rates (by

campus, by ethnicity, first generation) as compared to SAT/ACT?

Q2. What is the current evidence that Smarter Balanced assessment scores, either

alone or when paired with HSGPA, predict first-year college outcomes (GPA,

persistence to year 2) for UC students as compared to SAT/ACT?

Q3. Could a higher Smarter Balanced assessment score improve the probability of

admission of students from underrepresented groups and those who would be the

first in their families to attend college? For example, what are the admission rates for

students with lower HSGPA and higher Smarter Balanced assessment scores,

disaggregated by campus, ethnicity and first-generation status?

Q4. What measures has the Smarter Balanced Assessment Consortium (SBAC)

taken to minimize any bias and disparities, at the item and instrument levels, for

students who are from underrepresented groups? Are those measures reasonable

and sufficient?

Meetings. The SBSG met eight times, on Zoom, between June and September of

2021. SBSG conferred with representatives from the UCOP offices of Institutional

Research and Academic Planning (IRAP) and Undergraduate Admissions, and it also

reviewed relevant data and evidence provided by these offices. SBSG also met with

representatives from the Smarter Balanced Assessment Consortium (SBAC), the

nonprofit collaborative that developed and oversees the SB assessment, to discuss the

nature of this assessment, SBAC’s efforts to assess and reduce bias, the suitability of

these assessments for UC admissions, and the potential to modify them to meet the

university’s goals. SBSG also met with admissions officers from UC Irvine and UCLA to

discuss the holistic admissions process, learn about the application review and

acceptance process this past year when SAT/ACT scores were not included in the

September 2021 3

application, and discover their views on the applicability and utility of including the SB

assessments in UC freshman applications. (Note, admissions officers from other

campuses were also invited, but their schedules did not allow them to attend.) SBSG

also devoted substantial time to the discussion of UC admissions goals and testing

more broadly.

Overview of the Report. The report begins with a brief description of the current

admissions situation at the UC. It then provides SBSG’s responses to each of the four

questions posed in the charge. Insights SBSG obtained from representatives from

SBAC, IRAP, and the UC admissions offices are included throughout the report. The

report concludes with the SBSG main recommendation, followed by several additional

recommendations for the university regarding college preparation and admissions.

A BRIEF ACCOUNT OF THE CURRENT UC ADMISSIONS SITUATION

The number of students who apply to the UC has increased substantially over the last

decade. In the fall 2021 admissions cycle, the first cycle following the removal of the

SAT/ACT scores, the number of freshman applications reached an all-time high, with

over 200,000 applications systemwide, compared to just over 172,000 in 2020

(https://www.ucop.edu/institutional-research-academic-

planning/_files/factsheets/2021/table-1.1-freshman-applications-by-campus-and-

residency.pdf). Although there are likely many reasons for this increase, the absence of

the SAT/ACT requirement in the application undoubtedly explains some of it. The

COVID-19 pandemic also occurred during this period along with its wide-ranging effects

on students’ educational experience, including greater use of pass/no pass grading,

which may have helped increase UC applications.

The number of admission offers was also at an all-time high this year, with over 130,000

admits and with 43% of the California admits from underrepresented groups. In 2020,

UC admitted just over 119,000 students, and 42% of the California admits were from

underrepresented groups (https://www.ucop.edu/institutional-research-academic-

planning/_files/factsheets/2021/fall-2021-admission-table-2-1.pdf). The number of low-

income students admitted also increased by 10% since 2020

(https://www.ucop.edu/institutional-research-academic-

planning/_files/factsheets/2021/fall-2021-admission-table-3.pdf). The admissions

process varied across the system, with each campus using high school Grade Point

Average (HSGPA) and some combination of the 12 other factors used in the

comprehensive review of applicants

(https://admission.universityofcalifornia.edu/counselors/freshman/comprehensive-

review/).The admissions officers from UC Irvine and UCLA reported that reading and

September 2021 4

evaluating applications was difficult this past year, but, in their view, the main challenge

was the large number of applications and not the absence of the SAT/ACT score.

What does the increase in applications tell us? As the data indicate, the university is

in high demand. How we meet this demand going forward is a matter of great concern.

California has many high achieving secondary students, and in recent years, UC has

been asked to serve more and more of them. As one of three public institutions of

higher learning in the State, UC has the responsibility to serve as many Californians as

possible who meet our criteria for admission. However, present capacity levels of the

institution, set by State budget allocations, mean that the university is unable to serve all

the California high school students who merit admission. This reality makes it necessary

to have an equitable admissions process that can identify students who are adequately

prepared for and can benefit from the opportunities offered at the university. Equity is

central to this effort, especially because educational attainment is one of the best

predictors of lifetime earnings for individuals and college degree attainment remains the

most effective means of ensuring social mobility. Also, as a state public institution, the

UC is obliged to create a student body that is representative of the demographic profile

of California.

Gaps in access to UC. The UC admission process must not only identify potential

students among a large number of qualified high school students in an equitable way, it

must do so in the context of persistent opportunity gaps in applicants’ prior schooling

contexts and experiences and achievement gaps in GPA and test scores among

applicants. These gaps are, in part, evidenced by systematic differences by income and

race/ethnicity on standardized measures of academic achievement (i.e., standardized

test scores) and high school GPA (HSGPA). On average, students who are

socioeconomically disadvantaged perform less well on achievement tests and HSGPA

than their more advantaged peers. On average, African American, Hispanic/Latinx,

American Indian, and Asian/Pacific Islander heritage students from poor family

backgrounds tend to perform less well on the tests and HSGPA compared to students

from other social groups. Finally, there are important intersections of race and income;

students from different racial/ethnic backgrounds have a differential likelihood of being

socioeconomically disadvantaged based on structured societal inequities.

The reasons for these performance disparities and opportunity gaps are clear and have

been studied extensively (e.g., Carter & Welner, 2013; Desimone & Long, 2010;

Michelmore & Dynarski, 2017). The educational experiences of disadvantaged students

differ markedly from that of their more advantaged peers, and this difference is reflected

in and maintained over the years of schooling. Educational settings in poor communities

have fewer resources to support student learning and academic growth. It is not

September 2021 5

surprising, then, that on average, students from less resourced schools tend to perform

less well than their counterparts who attend better resourced schools. The cumulative

effect of these differences, from kindergarten through high school, is profound and is

reflected in performance on standardized tests and HSGPA. When institutions of higher

learning rely heavily on these tests for admission, a student’s performance directly

affects the chances of being admitted, especially at institutions with highly competitive

applicant pools.

In other words, although standardized test scores and HSGPA are presumed to signify

an individual’s academic skills and readiness for the university, these scores also

reflect, to varying degrees, broader social and educational inequities in the school

context. Furthermore, opportunity gaps are notably exacerbated by accumulated

advantages over time experienced by high income and traditionally privileged students

from dominant backgrounds. Thus, as disadvantages are created over time throughout

the schooling process, so too are accumulated advantages. All in all, the idea that any

academic achievement measure, even measures that are standardized and

administered in a consistent manner across test takers, can circumvent the broader

societal context in which academic opportunity and achievement are inextricably linked

should be met with skepticism.

The Smarter Balanced assessment. The SB assessment is a standardized test based

on the Common Core curriculum (a subset of the A-G course content for UC) given

once per year to public school children in grades 3 to 8 and grade 11. It was developed

and is overseen by the nonprofit Smarter Balanced Assessment Consortium (SBAC)

that works in partnership with boards of education, schools, and classroom teachers in

several states. Its purpose is to inform schools and teachers about how well their

students are meeting the learning goals of the Common Core curriculum. Unlike high-

stakes tests that have direct consequences for the test taker, the SB assessment is a

low-stakes test because the information it yields is used as feedback to parents,

students, teachers, schools, and school districts.

From the outset, the SBSG, like the preceding Feasibility Study Work Group, had

several reservations about the appropriateness of this assessment for university

admissions purposes.

First, as previously noted, the SB assessment measures how well classes or schools

are performing relative to benchmarks in the Common Core standards. As a result, its

use for UC admission would require a reframing (or expansion) of the assessment

beyond a K-12 accountability tool, and, thus, it seems likely to us that it would distort the

utility and validity of the assessment.

September 2021 6

Second, the use of the SB assessment to help decide on an individual student’s

admission to the UC would transform it from a low-stakes to a high-stakes test, and

problems associated with high-stakes testing would likely ensue. These problems

include, but are not limited to, worries and anxieties by students about their test

performance and efforts by schools, students, and families to optimize student

performance through test preparation and other means. Test preparation and related

efforts are time consuming and expensive, and because they exacerbate social

inequities, they end up reducing the access of disadvantaged students to the university.

Finally, the SB assessment, like other standardized tests, reflects and reproduces

inequalities and opportunity gaps in the K-12 system that, in turn, disadvantage

students in lower-income and underrepresented groups. As noted above as well as in

the Feasibility Study Steering Committee report and the Standardized Testing Task

Force report, students at lower-resourced schools have unequal access to opportunities

to learn which directly impacts their performance on standardized tests.

Now we turn to the four questions posed in the charge to the SBWG. We begin with

Question 4 regarding bias in the SB assessment because it provides useful framing for

our responses to the remaining questions.

[Q4] What measures has the Smarter Balanced Assessment Consortium (SBAC)

taken to minimize any bias and disparities, at the item and instrument levels, for

students who are from underrepresented groups? Are those measures

reasonable and sufficient?

In answering this question, we follow the recommendation in the Standards for

Educational and Psychological Testing (American Educational Research Association,

American Psychological Association, and National Council on Measurement in

Education [AERA, APA, & NCME], 2014, p. 1) that “all professional test developers,

sponsors, publishers, and users should make reasonable efforts to satisfy and follow

the Standards and should encourage others to do so” (emphasis added).

The Standards provides a framework for evaluating if the SBAC took reasonable and

sufficient steps to minimize bias and disparities in scores for students from

underrepresented groups. The Standards will also be useful in understanding the

responses to the other three questions which are related to predictive and incremental

validity.

September 2021 7

According to the Standards, scores on commercial measures and measures that

contribute to making high stakes decisions, including college admissions, are expected

to meet rigorous psychometric standards of validity, reliability, and fairness. The

question of bias is specifically related to the issue of fairness, which is defined as “the

validity of test score interpretations for intended use(s) for individuals from all relevant

subgroups. A test that is fair minimizes the construct-irrelevant variance associated with

individual characteristics and testing contexts that otherwise would compromise the

validity of scores for some individuals” (AERA, APA, & NCME, 2014, p. 219). Thus,

fairness contributes to the validity of the inferences derived from test scores because it

ensures that test performance does not vary systemically based on an individual’s

social or demographic characteristics. Reliability is “the degree to which scores are free

of random errors of measurement for a given group” (AERA, APA, & NCME, 2014, p.

223). This psychometric dimension of a test indicates the extent to which an individual’s

scores are similar over time and contexts and, therefore, consistently indicate the

individual’s performance on the construct being measured, rather than the individual’s

performance on a single assessment only or a score that is subject to other factors

extraneous to the construct.

Thus, test scores that are not fair or reliable cannot yield valid inferences; in other

words, they are not informative regarding an individual’s performance on whatever the

measure purports to assess. Moreover, issues of fairness, reliability, and ultimately,

validity (“the degree to which accumulated evidence and theory support a specific

interpretation of test scores for a given use of a test” [AERA, APA, & NCME, 2014, p.

225]), do not come into play after one has a test score; rather, they must be addressed

from the beginning of the test development process.

In its 2010 report, SBAC, citing the 1999 version of the Standards, indicated a

commitment to developing a balanced assessment system that was “credible, fair, and

technically sound” (SBAC, 2010, p. 2). To this end, in developing the SB assessments

for California, the SBAC has taken steps to ensure equity at several stages in an

evidence-centered design (ECD) approach. Across the design and planning, item

writing and review, field testing, and operational reviews and interpretation stages,

SBAC engages in various steps (SBAC, 2021a, 2021b, 2021c). These steps include the

following:

● Recruiting multidisciplinary and diverse teams of educators and education

leaders as well as experts in content, technology, accessibility and equity,

psycholinguistics and English language learning, psychometrics, and students

with disabilities to inform all aspects of the test design, development, and delivery

process;

September 2021 8

● Using multiple approaches to mitigate bias by recruiting facilitators, item writers,

and item reviewers that represent a diversity of views and multidisciplinary

expertise and providing standardized bias and sensitivity training to these

individuals to decrease the probability that the items are biased. Additionally, all

items are reviewed for content and adherence to the bias and sensitivity

guidelines by a diverse group of educators;

● Conducting laboratory and field-based assessments with students, including

groups of students who are underrepresented in higher education, to ensure that

the tests are measuring construct-relevant materials;

● Further psychometric evaluation at the item level includes conducting differential

item functioning (DIF) analyses for groups by gender, ethnicity-race, English

learner status, disability status, and Title 1/non-Title 1 school status; and at the

test level, includes examining the test scores for all students and for

demographic subgroups to ensure that they are comparable across groups and

high enough for use in high stakes decision making.

Using these various methods, any items that exhibit bias are removed from the item

pool (SBAC, 2021a, 2021b). Thus, the process outlined by SBAC is in keeping with the

Standards (2014) and is reasonable and sufficient to minimize bias and disparities

including with regard to underrepresented groups (emphasis added).

However, the SBSG notes that these efforts are largely aimed at reducing bias at the

item level and that additional efforts to reduce bias and disparities at the level of the test

are limited. These test-level scores reflect the aforementioned opportunity and

achievement gaps, and the same pattern of score differences present in the SAT and

ACT manifest in the SB assessment, with similar predictive validity (Kurlaender &

Cohen, 2019). Thus, despite all of the processes SBAC has in place for developing their

assessments, “large, persistent differences in performance exist between different racial

and ethnic groups at the test level” (SBAC, 2021b, p. 8). This outcome is not surprising,

as the SB assessments are assessing the same constructs (i.e., student achievement in

core academic subjects) assessed by the SAT, ACT, and HSGPA, and the pattern of

differences in the SB scores mirrors the patterns in all of these other assessments.

[Q1] What is the current evidence that Smarter Balanced assessment scores,

either alone or when paired with GPA, correlate with UC freshman admission

rates (by campus, by ethnicity, first generation) as compared to SAT/ACT?

To answer this question, we relied on Analysis of Potential Use of Smarter Balanced

Scores in Admissions and Placement at the University of California by UCOP’s

Institutional Research and Academic Planning (IRAP, in progress).

September 2021 9

Descriptive data were provided to explore the 2016, 2017, and 2018 admissions years.

Looking at the most recent year of data we note a clear pattern in admit rates by the

four Smarter Balanced performance standards: Did Not Meet, Nearly Met, Met, and

Exceeded. Students who met or exceeded the California 11

grade performance

standards are more likely to be admitted to UC, when compared to students who nearly

met or did not meet standards. This pattern is consistent across gender, race/ethnicity,

first-generation status, income, LCFF school designation, and UC campuses.

When exploring admit rates by SB levels and HSGPA we note that the overall pattern of

admit rates by SB levels holds across different GPA bands (i.e., <3.78; 3.78–4.08; and

>4.08), but it is more pronounced at the lower GPA bands. That is, the difference in

admit rates by SB levels appears to be more prominent among lower GPA students.

This general pattern holds across gender and in most cases race/ethnicity. As

expected, admit rates among lower GPA students (<3.78) are lower than among higher

GPA students, and adjusting for GPA levels, those with higher SB scores are still more

likely to be admitted than are those with lower SB scores. This is generally the case for

all groups; however, we note some differences in magnitudes by race, that are more

pronounced among lower GPA students. For students that are in the lower GPA band

(<3.78) and whose SB scores exceeded standards, African American students have

admit rates of 49%, Asian students 59%, White students at 39%, and Latinx students at

50%.

Exploring these relationships by campus, we note that most of the UC campuses admit

very few students with GPAs less than 3.78; among those who do (i.e., Merced,

Riverside, and Santa Cruz), this pattern also holds. Conditional on GPA, students with

higher SB scores are more likely to be admitted. At higher GPA levels (>3.78), we still

note that SB scores seem to be correlated with admit rates at several of the UC

campuses (namely, Irvine, San Diego, Santa Barbara, and Santa Cruz).

Supplemental analyses by Kurlaender and Cohen (2021) predict UC admission on the

basis of HSGPA (weighted and unweighted), with SBAC and SAT, respectively. These

analyses reveal that both the SAT and SBAC increase the predictive power of

admission status, above and beyond HSGPA to a moderate degree, but the difference

between the contribution of the SBAC versus the SAT is negligible. Results are

remarkably consistent by student subgroups and across all of the UC campuses

(Kurlaender & Cohen, unpublished).

[Q2] What is the current evidence that Smarter Balanced assessment scores,

either alone or when paired with GPA, predict first-year college outcomes (GPA,

persistence to year 2) for UC students as compared to SAT/ACT?

We first discuss the results from the Kurlaender and Cohen (2019) study for GPA and

IRAP’s analysis mentioned above (IRAP, in progress). Then we turn to results for

student persistence to year 2. Following an overall summary of these results, we make

a conclusion regarding the question.

September 2021 10

Evidence Regarding GPA. In terms of predicting first-year GPA at UC campuses

(UCGPA), high school GPA (HSGPA), the SAT test (SAT), and the SB assessment are

all quite similar in terms of raw scores, with the SAT doing best (correlation of .57), and

HSGPA and SB scores being somewhat lower (.48 and .51, respectively). When

Kurlaender and Cohen (2019) controlled statistically for the contribution of several

related, and potentially confounding, factors (such as a high school quality index,

parental SES), the values of the correlations are slightly higher and closer together

among the three measures (HSGPA, .57; SB, .54; SAT, .58), which suggests the

measures are tapping similar student information.

When HSGPA is combined with each of the other two measures (SAT, SB score), the

correlations improve (concentrating now on the raw scores without the controls

mentioned above). Adding the SB score to HSGPA increases the correlation to .58, an

improvement of .10, while adding the SAT to HSGPA increases the correlation to .62,

an improvement of .14. In terms of change in the percent of variance accounted for, SB

score increases the explained variation from 23.0% to 33.6%, and SAT increases it to

38.4%.

Going beyond a combination of the HSGPA with each of the other measures, one can

consider the correlation of UCGPA with both of them, and the increase is quite small,

with SAT adding 4.8 percentage points to HSGPA+ SB score, while the SB score adds

nothing to the combination of HSGPA+ SAT.

Examining the results when looking across race/ethnicity, socioeconomic disadvantage,

and school quality, the results are much the same. There is a slight improvement in

correlation for SAT (compared to SB score) in terms of school quality.

We also examined the effect of HSGPA, SAT, and SB scores on subgroup

representation in the top 10% of the UC applicant pool. Looking first at student ethnicity,

compared to HSGPA, both SB scores and SAT result in (a) increases for Asian/Pacific

Islanders (from 37% to 55% and 61%, respectively), and decreases for White (35% to

31% and 29%, respectively), Hispanic/Latinx (22% to 9% and 5%, respectively), and

African American (2% to 1% and .06%, respectively) students. This pattern tells us that

using either test reduces the proportions of African American, Hispanic/Latinx, and

White students in the top 10%, while it increases the proportion of Asian/Pacific Islander

students. The same procedure can be carried out for socioeconomic disadvantage, and

the results are similar. Compared to HSGPA, both SB scores and SAT result in fewer

disadvantaged students in the top 10% (29% to 15% and 8%, respectively).

September 2021 11

Finally, when similar analyses were conducted for UCGPA, as seen in IRAP’s analysis

(IRAP, in progress), the overall results broadly reflect these results (though IRAP also

included the ACT Composite test score).

Evidence Regarding Persistence to Year 2. In terms of persistence of students to

year 2 of their studies, the overall results from the Kurlaender and Cohen (2019) study

are largely consistent with those for UCGPA (IRAP, in progress). That is, the SB score

does just about as well as the SAT, either alone, or when paired with HSGPA. The main

point though is that none of these three measures is a strong predictor of year 2

persistence. However, this might be due to a “ceiling” effect — that is, 93% of UC

students do indeed persist into second year, so there is not much variation that could be

sensitive to any student measure.

These results indicate that all three measures (SAT, SB score, HSGPA) do about

equally well in predicting first-year UCGPA. More specifically, the SB score does about

as well as either HSGPA or SAT in predicting a student’s first-year GPA at the UC.

When considered as additive to HSGPA, both SB scores and SAT add a moderate

amount of information, and, again, about the same amount (i.e., the SB score does

about as well as the SAT in increasing explanatory power). Looking at subgroups, in

terms of representation in the predicted top 10% of the UC applicant pool, students with

more socioeconomic disadvantage and students from underrepresented groups fare

considerably better when using just HSGPA than when it is combined with either the

score from SB score or SAT. In terms of persistence to year 2, the results are much the

same, though somewhat less conclusive, as none of the measures predict even

moderately.

Our conclusion regarding the specific question is that SB scores, either alone or when

paired with HSGPA, predict first-year college outcomes (GPA, persistence to year 2) for

UC students about equally as well as the SAT.

[Q3] Could a higher Smarter Balanced assessment score improve the probability

of admission of students from underrepresented groups and those who would be

the first in their families to attend college? For example, what are the admission

rates for students with lower GPA and higher SB scores, disaggregated by

campus, ethnicity and first-generation status?

The answer to this question is necessarily speculative for two reasons. First, since the

UC has never considered SB scores in admissions decisions, the currently available

data cannot tell us how those scores would be used in admission decisions. Second,

existing data also cannot tell us how higher SB scores might affect the probability of

September 2021 12

admission under an admissions process that does not use the SAT or ACT score.

Although the cohort of applicants for Fall 2021 was evaluated without SAT or ACT

scores, this cohort also did not take the SB assessment in 11th grade due to the

COVID-19 pandemic.

Data from previous admissions cycles do tell us that students with higher SB scores are

sometimes admitted despite having a HSGPA that is lower than most other admitted

applicants. However, it is unclear why these students were more likely to be admitted. It

is likely due, at least in part, to the strong correlation between SB scores and SAT

scores and the role that SAT historically played in admissions. However, it could also be

driven by other qualifications that cause students with high SB scores to receive higher

evaluations, independent of their SAT scores.

If we assume that the positive correlation between the SB assessment and SAT is the

primary explanation for the higher admit rates of students with higher SB scores among

those with lower HSGPAs—and if we further assume that SB scores would be used in

admissions decisions similarly to the way SAT scores were used in past years—then we

can conclude that the use of SB scores in admissions would likely tend to benefit

groups of students with historically higher average SAT scores. Specifically, while it

would improve the probability of admission for some students from historically excluded

groups and first-generation students, it would disproportionately benefit Asian/Pacific

Islander students and students from higher-income families, while reducing admission

rates of African American, Hispanic/Latinx, and low-income applicants.

To be sure, considerations discussed elsewhere in this report are also relevant to this

question. In particular, there are some reasons to believe that the SB assessment (as

compared to the SAT) might provide a better and more equitable tool for identifying

students with lower HSGPAs but high potential to succeed academically. First, it is

possible that, at least initially, SB scores would be less influenced by extracurricular

coaching and test-preparation since the SB assessment is designed to test the

Common Core State Standards taught in all California public schools (but this might be

modified over time). Second, since all California public school students take the SB

assessment in 11th grade, informational tools could be developed to encourage

students with high scores to apply. Third, the availability of SB scores for all California

public school students could make it easier for admissions committees to evaluate the

scores in the local context of each student’s high school. By contrast, it is harder to

adjust SAT scores for local context because in lower-performing schools, only the very

highest-scoring students submit applications to the UC.

As to the identification of the top 10% of students in the UC applicant pool, Kurlaender

and Cohen (2019) find that when SB scores are combined with high school GPA to

September 2021 13

predict first-year GPA among UC freshmen, lower-income applicants and those from

historically excluded groups are somewhat more likely to be represented in the top 10%

of the UC applicant pool when SB scores replace SAT scores in the prediction model.

That said, predicting first-year UCGPA based on high school GPA alone yields the most

diverse top 10% pool in terms of lower-income and historically excluded groups of

students.

At the same time, there are reasons to doubt that use of the SB assessment would lead

to improved admissions probabilities for students from historically excluded and

disadvantaged groups. If used in the UC admissions process, the SB assessment would

become a high-stakes test. For this reason alone, the distributions of scores – in

particular by income, race, ethnicity and gender – are likely to become more similar to

those seen in the SAT. For example, as discussed elsewhere in this report, the

introduction of any new test in the UC admission process is likely to be followed by the

development of test-preparation courses and coaching services in the private market, to

the advantage of students with greater family resources.

In sum, there is no guarantee that use of the SB assessment in UC admissions would

lead to higher admission rates of students from historically excluded groups and those

who would be the first in their families to attend college. However, the question of how

best to identify students with high potential for success but lower HSGPAs – especially

those from disadvantaged backgrounds – is an important one that deserves further

attention. Data from future cohorts of UC applicants would allow a formal analysis of

how dropping the SAT from the admissions process affected the admission rates of

students with higher SB scores but lower HSGPAs, including detailed breakdowns of

this impact across campuses and student demographics. The committee strongly

recommends that the UC continuously evaluate how the move to test-free admissions is

affecting patterns of applications, admissions, and student success. This work could be

done through a sub-committee of BOARS in coordination with IRAP.

CONCLUSIONS AND RECOMMENDATIONS

Based on our analyses and deliberations, the SBSG believes that the SB assessment is

not appropriate as an admissions test, required or optional, for the UC. Data show that

SB 11

grade test scores add only modest incremental value beyond HSGPA in

predicting first-year grades, and would likely come at the same cost as the SAT. That is,

similar to the SAT, the SB assessment captures the inequities in opportunities to learn

across California schools that are pronounced by race and socioeconomic status.

Moreover, converting SB from a low-stakes assessment intended to measure student

achievement for school accountability into a high-stakes test that would impact college

admissions decisions for individual students is likely to lead to the development of SB

September 2021 14

test preparation ventures, similar to those currently in place for the SAT/ACT, that have

been shown to magnify score differences among demographic groups. Such test

preparation development could also undermine the purpose and current use of the

assessment in K-12. More generally, we have strong reservations about the ability of

any test of A-G course content to remediate educational inequities experienced by K-12

students or to attenuate the unequal opportunities for college preparation in under-

resourced schools.

UC had an all-time high number of applications and a substantial increase in California

freshman admits for the 2021-22 entering class. Although the impact of the global

pandemic on college application choices is unclear, the elimination of standardized tests

is likely a factor in UC receiving more applications overall, including from

underrepresented and low-income students. We strongly recommend that UC continue

to monitor comprehensive review in admissions across the campuses and their use of

the 13 criteria, particularly academic achievement indicators (e.g., availability of

weighted courses) in selecting students for admission.

The SBSG affirms that there is value in understanding the SB assessment and how well

it helps prepare K-12 students for postsecondary school, particularly at UC. We envision

a stronger partnership with K-12 schools, one that connects the UC more closely with

the learning opportunities and academic support offered in CA high schools.

California has many high achieving high school students and in recent years UC has

been asked to serve more and more of them. Capacity levels set by State budget

allocations keep the university from admitting all of the interested and deserving

students in the State. Therefore, it is necessary that the university use a fair and

transparent method of determining who to admit to the UC. Going forward, SBSG

stresses that it is important for the university and the community at large to understand

that there is not one thing, such as a change in the admissions process, forms of

support, or additional resources, that will enable us to reach our equity goals in UC

admissions. Rather, multiple strategies are needed. To this end, we offer the following

additional recommendations.

ADDITIONAL RECOMMENDATIONS

Build a Stronger Partnership with K-12. UC should continue to play a role in

advancing educational equity in college preparation by forging a stronger academic

partnership with K-12.

September 2021 15

● Strengthen and expand high school programs that increase student access to

and success in the UC A-G requirements. All K-12 students in the State should

have adequate access to and support for completing the UC A-G requirements,

which will also help expand the geographic diversity of UC undergraduates.

● Advocate for permanent legislative funding for Student Academic Preparation

and Educational Partnerships (SAPEP).

● Collaborate with California high schools to monitor applications and admissions

growth by high school to determine if the university is reaching students from

underserved high schools.

● Encourage all California high schools to implement the SB interim assessments

as a formative evaluation tool. These assessments, along with the current 11

grade summative assessment, provide important information to students on their

preparation for college.

● Work with SBAC, California State Board of Education (SBE) and California high

schools to expand the SB assessment item bank to include more challenging

items in core subject areas and also help them improve and expand the

performance tasks to be more useful for teachers and students regarding college

preparation.

● Investigate the use of the SB assessment as one of multiple measures used for

writing placement after students enroll at UC.

Bolster the Holistic Review Process. The UC application contains significant

information about students’ academic experiences. However, it is time consuming to

extract and evaluate this information fairly in the context of opportunities at the student’s

high school. This process can be aided by the following.

● Encourage and provide adequate support and resources so that all nine

campuses can use holistic review for freshman admissions.

● Develop local expertise among readers of the applications regarding the use of

contextual information and provide annual anti-bias training for all readers.

● Work closely with BOARS in the continuing development and implementation of

the holistic review process.

September 2021 16

Expand and Develop Resources for Students After They Enroll at UC. There are

benefits for students and savings for the campus when the university invests in

resources to support student academic success after they enroll at the university.

● Fortify effective campus programs that support student success with sufficient

resources and outreach to continue to help students advance to degree.

● Share information across the campuses about effective student support

programs, including those within academic departments, and provide resources

to campuses that want to develop these programs.

● Monitor these programs according to a common set of outcomes to ensure they

are meeting student success goals.

Research the Efficacy of the Admissions Process and Student Success. There are

many important research questions about admissions that the university will need to

address in the coming years. This research should be both quantitative and qualitative,

and conducted in collaboration with the Academic Senate. Research topics should

include the following.

● A subcommittee of BOARS that includes content experts in the areas of

admissions and racial equity, should engage in regular monitoring of the

outcomes of test-free admission to determine the impact on admissions and

student success, including freshman GPA, first-year persistence, probation rates,

graduation, and time to degree. Potentially contributing factors for the initial

cohorts need to be taken into account, including remote learning, allowance for

credit/no credit grades, limited access to counselors/college advising, and

students’ use of institutional supports.

● Investigate how students are faring when they get to UC. The analyses need to

take into account various factors considered critical to student success at UC,

including the campus climate, types and extent of academic preparation before

students enroll, student’s utilization of campus learning resources, and how

different academic programs guide and support students in their major area of

study.

● Qualitative study of many aspects of the admissions process is critical, including

historical analysis of how this process has contributed to the disenfranchisement

of communities of color at the University and detailed study of why high achieving

September 2021 17

students from underrepresented groups who are admitted to the UC choose to

go elsewhere.

Capacity Limitations. A long-range concern is how the university can step up as a

system to increase capacity and serve more California undergraduate students. To

sustain the academic stature and excellence of the UC, this effort will require much

more than teaching additional courses. Academic programs, especially those in high

demand, need to be expanded and new programs need to be developed to address

emerging research issues and meet pressing societal needs and student interests.

These activities depend on the vision and effort of Senate faculty who are charged with

overseeing and developing the academic programs of the university, which leads

directly to the following recommendations.

● The UC needs more state funding to hire tenure-track faculty both to teach and to

develop programs to match this new growth in the student body.

● More academic activities and resources outside the classroom that are

appropriate and expected of an R1 institution need to be developed and

supported.

● Additional resources and support for academic program development need to be

provided to campuses that are poised for growth.

References

American Educational Research Association, American Psychological Association, &

National Council on Measurement in Education. (1999). Standards for

educational and psychological testing. American Educational Research

Association.

American Educational Research Association, American Psychological Association, &

National Council on Measurement in Education. (2014). Standards for

educational and psychological testing. American Educational Research

Association.

Carter, P. L., & Welner, K. G. (2013). Closing the opportunity gap: What America must

do to give every child an even chance. New York, NY: Oxford University Press.

https://doi.org/10.1093/acprof:oso/9780199982981.001.0001

Desimone, L. M., & Long, D. (2010). Teacher effect and the achievement gap: Do

teacher and teaching quality influence the achievement gap Between Black and

White and high- low-SES students in the early grade? Teacher College Record,

12, 3024–3073.

September 2021 18

Feasibility Study Steering Committee. (2020). Recommendation of the Feasibility Study

Steering Committee to UC President Drake.

https://regents.universityofcalifornia.edu/regmeet/jan21/b2attach7.pdf

Institutional Research and Academic Planning at UC Office of the President. (in

progress). Analysis of Potential Use of Smarter Balanced Scores in Admissions

and Placement at the University of California.

Kurlaender, M., & Cohen, K. (2019). Predicting college success: How do different high

school assessments measure up? Policy Analysis for California Education.

Kurlaender, M., & Cohen, K. (unpublished). Supplemental Analysis for Smarter

Balanced Study Group.

Michelmore, K., & Dynarski, S. (2017). The gap within the gap: Using longitudinal data

to understand income differences in educational outcomes. AERA Open, 3(1).

https://doi.org/10.1177/2332858417692958

Standardized Testing Task Force (STTF; 2020). Report of the UC Academic Council

Standardized Testing Task Force (STTF).

https://senate.universityofcalifornia.edu/_files/committees/sttf/sttf-report.pdf

Smarter Balanced Assessment Consortium. (2010). Theory of action: An excerpt from

the Smarter Balanced Race to the Top application.

https://files.eric.ed.gov/fulltext/ED536956.pdf

Smarter Balanced Assessment Consortium. (2021a). Equity and Smarter Balanced:

Assessing fairness through test construction.

Smarter Balanced Assessment Consortium. (2021b). Smarter Balance Assessment

Consortium: Bias and sensitivity guidelines.

https://portal.smarterbalanced.org/library/en/bias-and-sensitivity-guidelines.pdf

Smarter Balanced Assessment Consortium. (2021c). Usability, accessibility, and

accommodations guidelines.

https://portal.smarterbalanced.org/library/en/usability-accessibility-and-

accommodations-guidelines.pdf

September 2021 19

Attachment

SMARTER BALANCED STUDY GROUP ROSTER

Mary Gauvain, Co-Chair

UCR, Professor, Psychology

Madeleine Sorapure, Co-Chair

UCSB, Professor, Writing Program

Eddie Comeaux

UCR, Professor, Education

Frances Contreras

UCSD, Professor, Education

Laura Giuliano

UCSC, Professor, Economics

Uma Jayakumar

UCR, Professor, Education

Michal Kurlaender

UCD, Professor, Education

Mark Wilson

UCB, Professor, Education

Frank C. Worrell

UCB, Professor, Education

Anne Zanzucchi

UCM, Professor, Writing Program

Staff to the Study Group

Abigail K. Bates

UCOP, Senior Research Analyst

Kari Stewart

UCOP, Executive Director, Academic Success Initiatives