December 1, 2020
Comment Intake
Consumer Financial Protection Bureau
1700 G Street, NW
Washington, DC 20552
Re: Request for Information on the Equal Credit Opportunity Act; Docket No. CFPB-2020-0026
Dear Director Kraninger:
Thank you for the opportunity to comment on the Consumer Financial Protection Bureau’s (CFPB) Request For Information (RFI) on the Equal Credit Opportunity Act (ECOA).
The National Community Reinvestment Coalition (NCRC) consists of more than 600 community-based organizations, fighting for economic justice for almost 30 years. Our mission is to create opportunities for people and communities to build and maintain wealth. NCRC members include community reinvestment organizations, community development corporations, local and state government agencies, faith-based institutions, fair housing and civil rights groups, minority and women-owned business associations, and housing counselors from across the nation. NCRC and its members work to create wealth opportunities by eliminating discriminatory lending practices, which have historically contributed to economic inequality.
As an organization devoted to promoting fair lending practices, we are especially interested in this RFI as ECOA, an anti-discrimination legislation, has jurisdiction over all forms of lending compared to the Fair Housing Act, which is limited in its scope. ECOA is a critical tool used to identify and address discrimination in credit transactions based on race, color, religion, national origin, sex, marital status, age, and other protected statuses. Accessing safe and responsible credit products is vital for the economy’s growth, especially now that the COVID-19 pandemic has crippled the economy.
NCRC requests that the CFPB provide further guidance on:[1]
Question 1:
- the disparate impact standard by updating the language to state that the creditor practice must meet a “substantial, legitimate, nondiscriminatory interest.”
- stating that lenders are responsible for algorithms and predictive models that have a disparate impact on members of protected classes, even if they are created or maintained by third parties.
Question 2:
- creating a national interpreter and translator certification program.
- provide glossaries in other languages and Language Access Plan (LAP) template for lenders
Question 3:
- how to navigate Special Purpose Credit Programs (SPCP) including but not limited to acceptable categories of SPCPs and second-look programs.
Question 4:
- how to reach out to new markets without harming other protected classes.
- how to reach out to people with limited English proficiency.
Question 5:
- increasing its enforcement actions in the small business lending sphere.
- moving forward with a strong proposed Section 1071 proposed rule in 2021 and issuing a final rule in 2022.
Question 6:
- interpreting the protected class of sex within ECOA with the same understanding as the Court does in Bostock.
Question 8:
- how any category of public assistance that is verified but does not include a specific duration should be accepted by the creditor and considered for ability to pay.
Question 9:
- how accommodations in AI models can be reformulated to preference gains in fairness – even if the new iteration forces lender to make a modest concession to assessments of default risk.
- the use of monotonic constraints in AI/ML modeling.
- testing data sets used by lenders for their consistency with valid real-world lending patterns.
- stress-testing AI models for their predictive power during crises.
- reviewing how cultural bias within a lending institution could bias the outcomes of its modeling.
Question 10:
- creating adverse action notices that are accurate, explainable, interpretable, and actionable.
- on how to create notices that overcome the problem of collapsing multiple related variables into a single reason code.
- the use of Shapley values to identify the most significant characteristics within a borrower’s application that led to an adverse action.
- how digital formats could use better graphic design – including expandable accordion structures – to enhance the ability of consumers to understand adverse action notices.
Additional Ideas:
- Provide further guidance on an interagency basis with the federal bank regulatory agencies to apply its updated ECOA guidance to fair lending reviews that are concurrent with CRA exams.
Question 1: Disparate Impact
Should the Bureau provide additional clarity regarding its approach to disparate impact analysis under ECOA and/or Regulation B? If so, in what way(s)?”
Disparate impact is a recognized behavior of discrimination under ECOA. Providing clarity into the use of the disparate impact standard has become especially crucial since the Department of Housing and Urban Development (HUD) issued a final rule that narrows the applicability of the standard in cases arising under the Fair Housing Act.[2] HUD’s new rule puts an onerous burden on plaintiffs, and its approach should not be confused with the CFPB’s approach to disparate impact.
The CFPB needs to revise the language of Regulation B to make it more straightforward. The current language of Regulation B states that a creditor practice that is discriminatory in effect may be prohibited “unless the creditor practice meets a legitimate business need that cannot reasonably be achieved as well by means that are less disparate in their impact.” The CFPB should update this language to state that the creditor practice must meet a “substantial, legitimate, nondiscriminatory interest.” HUD used this phrasing in the 2013 disparate impact rule.[3] Adding the words “substantial” and “nondiscriminatory” strengthens the message about what standard is necessary to justify a discriminatory practice.
We also want the CFPB to update the example of how to evaluate a creditor practice for disparate impact. The current example states that an income requirement for credit applicants could result in women and minority applicants being rejected at a higher rate than men and nonminority applicants, but adds that “if there is a demonstrable relationship between the income requirement and creditworthiness for the level of credit involved, however, use of the income standard would likely be permissible.” This example does not illustrate the possibility that another practice with a less disparate impact might achieve the same business need. The example would be more effective if it advised that the lender consider other options with less discriminatory results.
Disparate Impact and Algorithms
The CFPB should also clarify that when a lender relies on an algorithm or predictive model created or maintained by a third party, the lender is still legally responsible for any results that cause a disproportionately negative impact on a prohibited basis. There has been significant confusion about this issue, in part because HUD’s proposed disparate impact rule from 2019 included a defense available to respondents who showed that a third party was responsible for their algorithm.[4]
The final rule removed this defense in favor of a broader defense. However, the final rule contains other defenses related to algorithms that produce discriminatory results, such as a defense for housing providers that can show that their predictive analysis accurately assesses risk.[5]
The CFPB needs to clearly state that lenders are responsible for algorithms and predictive models that have a disparate impact on members of protected classes, even if they are created or maintained by third parties. This guidance would be consistent with guidance from the prudential regulators that lenders are responsible for ensuring their third-party vendors’ compliance.[6] This action would encourage lenders to take responsibility for their practices and consider less discriminatory alternatives to models that disproportionately burden members of protected classes. It would also eliminate confusion surrounding the use of these models and algorithms. It may also provide an incentive for lenders to use machine learning and artificial intelligence to find methods, such as adversarial debiasing, that will make their models more inclusive.
Question 2: Limited English Proficiency
The Bureau seeks to understand the challenges specific to serving LEP consumers and to find ways to encourage creditors to increase assistance to LEP consumers. Should the Bureau provide additional clarity under ECOA and/or Regulation B to further encourage creditors to provide assistance, products, and services in languages other than English to consumers with limited English proficiency? If so, in what way(s)?
In November 2017, the CFPB published a Spotlight on serving limited English proficient (LEP) consumers.[7] This spotlight highlighted practices and resources that some institutions were providing to LEP customers. The practices highlighted within that report are best practices that other institutions should implement. Unfortunately, this report does not give any specific guidance to protect financial institutions from ECOA violations while expanding the credit marketplace to this particular consumer group. The CFPB should provide additional guidance on how to expand into this market.
The Spotlight report reveals that there is no nationally-recognized organization that offers financial interpreters and translator certification. The creation of a national interpreter and translator certification program will correct this void. Currently, this void creates entry barriers into the credit marketplace as consumers do not have the language skills to understand how to navigate the credit market. The CFPB should administer oversight of this program. The financial interpreter and translation certification program should be for the top ten foreign languages spoken in the United States. It should also include cultural differences; for example, Spanish should cover all different dialects such as Mexican and Cuban Spanish. The program’s implementation should be similar to the housing counseling certification program that the US Department of Housing and Urban Development oversees. This type of national certification program will bring uniformity to the specific language used across the country and provide for the consistent translation and understanding of financial terms. This uniformity can better inform borrowers and create a more even playing field across the lending marketplace.
Glossaries and Templates
The CFPB should also provide glossaries in other languages as it did with Spanish and Chinese.[8] Providing these glossaries and producing financial literacy toolkits in languages other than English will foster the uniformity of translation and increase financial literacy.
The CFPB could also provide a Language Access Plan (LAP) template for lenders. The CFPB offered some guidance when it published its LAP in 2017.[9] Although the guidance included the four core considerations the CFPB used to develop its plan, a template for lending institutions that provides guidance on the information lending institutions should include in their LAPs, would allow the CFPB to more easily compare LAPs from different lenders. It would also give more explicit guidance to lenders on the elements of a reasonable LAP and ensure that a lender’s approach and products offered addressed their LEP customers’ and applicants’ needs.
Question 3: Special Purpose Credit Programs
Should the Bureau address any potential regulatory uncertainty and facilitate the use of SPCPs? If so, in what way(s)? For example, should the Bureau clarify any of the SPCP provisions in Regulation B?
ECOA states that a for-profit lender can extend special purpose credit to an applicant and not to others as long as it “meets the standards prescribed in regulation by the CFPB.”[10] NCRC supports the CFPB clarifying its standards to facilitate greater use of special purpose credit programs (SPCPs) because currently, these programs are underutilized due to regulatory uncertainty. The section of Regulation B related to standards for SPCPs,[11] conveys only high-level guidance, which has made it difficult for lenders to lean in without fear of violating provisions of ECOA and other fair lending laws.
The CFPB and the Office of the Comptroller of the Currency (OCC) have provided a few examples of acceptable categories of SPCPs in the past;[12] however, more transparency and guidance are needed. What constitutes an acceptable written plan and what types of substantiation can be submitted to support the extension of an SPCP to a “class of persons” could be clarified with more explicit guidance.[13]For example, a written report template would allow lenders flexibility for innovation while also providing lenders with a format, guardrails, and directions to meet CFPB standards. A template would also make it easier for agencies with Community Reinvestment Act (CRA) authority to compare lenders’ plans. Lenders would also benefit from additional guidance from previous CFPB exams on any concerns about who constitutes a class that would ordinarily not have been extended credit.
The CFPB should specifically weigh in on the use of second-look programs, which give denied applicants another chance at qualifying for credit, and include standards for what constitutes acceptable public data to substantiate the definition of a class of persons eligible for an SPCP.
The CFPB should also address the regulatory uncertainty that has discouraged many lenders from pursuing SPCP opportunities. As part of its interagency discussions related to fair lending enforcement, the CFPB should take the lead in ensuring that its SPCP guidance under ECOA is consistent with the approach that other federal regulators, the Department of Justice, and the state attorneys general are taking in enforcing other fair lending laws.[14] The lending community should not be discouraged from participating because regulators have different compliance requirements in their fair lending enforcement approach.
The guidance could also provide greater detail on any concern the CFPB has about an SPCP favoring applicants with one or more common characteristics that may restrict applications from other groups. The current guidance from the regulators has only addressed the most overt forms of discrimination. It would help the lending and advocacy communities better understand the guardrails to avoid less blatant but still harmful forms of discrimination that might result from a well-intentioned SPCP program.
Implementation of SPCP
The CFPB should also aid the lending community in navigating the complexities of implementing an SPCP by encouraging greater transparency. By creating office hours specifically for SPCP program discussions with lenders and the advocacy community, the CFPB can foster greater transparency. The CFPB should encourage lenders to provide an overview of their written plans and their substantiation for identifying a class of persons to Fair Lending and Office of Innovation staff. This would help the CFPB identify any concerns and gain suggestions for an even tighter fit between the SPCP and the needs of underserved communities.
Finally, the CFPB and lenders should not lose focus in their discussions of SPCPs on the long-term impacts of a program. An SPCP should be vigilant not to unintentionally foster gentrification or perpetuate segregation patterns, even though it may aid an underserved population in the short term. For example, lenders should focus on a particular applicant’s specific characteristics in determining eligibility for an SPCP and not geography, even though it would be acceptable for the lender to provide additional outreach in communities of color. The CFPB guidance on using advertising to reach underserved communities should be consistent with its approach on SPCP so that outreach campaigns and eligibility requirements encourage social and geographic mobility.
Question 4: Affirmative Advertising to Disadvantaged Groups
The official interpretation to Regulation B states that “[a]creditor may affirmatively solicit or encourage members of traditionally disadvantaged groups to apply for credit, especially groups that might not normally seek credit from that creditor.” The Bureau understands from its stakeholder engagement that creditors are interested in additional guidance that may be helpful to them in developing such marketing campaigns while ensuring regulatory compliance. Should the Bureau provide clarity under ECOA and/or Regulation B to further encourage creditors to use such affirmative advertising to reach traditionally disadvantaged consumers and communities? If so, in what way(s)?
Targeted Advertising
Targeted advertising is when an advertiser delivers relevant advertisements to a specific clientele based upon that clientele’s characteristics. As data collection has expanded, advertisers have more data to mine, thus better targeting their ads to their intended audiences. Fair lending concerns arise if an institution chooses to filter its display advertisements to provide different and potentially suboptimal offerings to different protected class members. This advertising can result in the positive effects of affirmative advertising to one protected group while simultaneously having a disparate impact or treatment on another protected group. Furthermore, financial institutions need to be held responsible for the algorithms used by third-party advertising platforms. There is no difference between marketing companies and other service provider[15] that financial institutions engage with.
The CFPB needs to provide additional guidance on affirmative advertising so that financial institutions can reach out to new markets without harming other protected classes.
Limited English Proficient Consumers and Affirmative Advertising
In 2017, the CFPB published a Spotlight on interactions with the LEP consumers.[16] This Spotlight report neglected to focus on affirmative advertising for this group of disenfranchised consumers. The CFPB needs to provide guidance specifically on affirmative advertising to this sector of consumers without violating ECOA guidance. This guidance should include actions that financial institutions can implement.
Question 5: Small Business Lending
In light of the Bureau’s authority under ECOA/Regulation B, in what way(s) might it support efforts to meet the credit needs of small businesses, particularly those that are minority-owned and women-owned?
Enforcement Actions
The COVID-19 pandemic has resulted in the closure of at least 100,000 small businesses.[17] One of the reasons for these closures was that these businesses could not access capital. Congress created the Paycheck Protection Program (PPP) to provide a grant-like incentive for small businesses to keep their employees on payroll and provide support for business mortgages, rent, and utilities during the COVID-19 pandemic. There were two rounds of PPP funding; however, not all of the money devoted to this program was expended.
While accessing PPP funding during the first round of the program was challenging for all businesses, there is evidence that businesses owned by people of color and located in communities of color were less likely than white businesses to access PPP funds.[18]
Further research suggests that small business owners of color [19] and businesses in communities of color[20] faced challenges applying for and approval for PPP funding. More detailed, loan-level analysis is challenging; however, due to frequent missing data on the race, ethnicity, gender, and veteran status of the borrower available in the PPP data released to date. SBA program loan forms give borrowers the option to disclose race, ethnicity, gender, and veteran status, although this disclosure is voluntary. However, in the collection of PPP data, the SBA excluded optional standard demographic variables in the initial application forms. While these variables were eventually included, the delay and optional disclosure requirements meant that only 23 percent[21] of the loan records had race or gender data.
Twice during the pandemic, NCRC engaged in mystery shopping for small business loans in the Washington DC and Los Angeles MSAs. We tested in the Washington DC MSA at the beginning of the second round of PPP lending and in Los Angeles during the end of PPP funding.
Washington DC Testing Results
Our findings in Washington DC revealed statistically significant disparities between testers’ groups using the chi-square difference test across the marketplace. We found:
- A difference in levels of encouragement in applying for a loan
- A difference in the products offered
- A difference in the information provided by the bank representative
A fair lending analysis of matched-pair tests found that there was a difference in treatment in 27 out of 63 (43%) tests, with the White tester receiving more favorable treatment than the Black tester in the small business pre-application arena in violation of ECOA. Further analysis revealed that Black testers experienced differences in treatment through a difference in the tester’s information requested. In two different tests, Black male testers were offered a home equity line of credit (HELOC) products instead of/in conjunction with small business loan products. In 12 out of the 27 control favored tests (44%), we identified disparate treatment as a part of our fair lending review. Lenders not only discouraged the Black testers from applying for a loan but simultaneously encouraged similarly situated White testers to apply for one or more loan products. This “double impact” on minority applicants, discouragement, and failure to provide complete information not only limits minority access to credit it also damages the credibility of the small business lending community.
Of the 17 different financial institutions tested in this audit, 13 institutions had at least one test that showed the control tester was favored. Black females received worse treatment in 59% of the tests we found a difference in treatment. Overall, there were 49 different instances of discrimination that the Black testers experienced.
Los Angeles Testing Results
Our analysis revealed statistically significant differences in how the male and female testers were treated throughout the interaction. Our chi-square analysis revealed that the Black female testers across the marketplace as a whole were treated significantly worse compared to the Hispanic and White female testers, while the difference in treatment for male testers only rose to significance in a few areas.
An overall fair lending analysis of all the individual male and female matched-pair multi-layered matched tests found that a combined 21 out of 60 (35%) tests revealed a difference in treatment with the White tester favored over either or both of the Black and Hispanic Testers in violation of ECOA. There were 57 different instances of discrimination in this round experienced by the Black and Hispanic testers.
More specifically, in the female multi-layered matched tests, we found that 16 out of 30 (53%) tests revealed a difference in treatment based on race and/or national origin. When we look at the specific tests by protected class, we found that the Black females received worse treatment in 46% of the female tests, and the Hispanic females received worse treatment in 36% of the female tests (Figure 1). There were 44 different instances of discrimination that the Hispanic and Black female testers experienced.
Figure 1: Testing revels Black and Hispanic female testers faced worse treatment than White female testers
| Female Race Tests (White v Black) | Female National Origin Tests (White v Hispanic) | ||
| Control Favored | 14 | Control Favored | 11 |
| Overall Tests | 30 | Overall Tests | 30 |
| Percentage | 46% | Percentage | 36% |
Source: Lederer, A & Oros, S. November 12, 2020.Lending Discrimination During Covid-19: Black and Hispanic Women Owned Businesses” NCRC. https://ncrc.org/lending-discrimination-during-covid-19-black-and-hispanic-women-owned-businesses/
In the individual matched-pair tests for male testers, we found that 5 out of 30 (16%) tests revealed a difference in treatment based on race and/or national origin. Black males received worse treatment in 16% of the male tests (Figure 2). Hispanic males received worse treatment in 13% of the male tests.
Figure 2: Testing reveals that Black and Hispanic male testers faced worse treatment than White male testers
| Male Race Tests (White v Black) | Male National Origin Tests (White v Hispanic) | ||
| Control Favored | 5 | Control Favored | 4 |
| Overall Tests | 30 | Overall Tests | 30 |
| Percentage | 16% | Percentage | 13% |
Source: Lederer, A & Oros, S. November 12, 2020.Lending Discrimination During Covid-19: Black and Hispanic Women Owned Businesses” NCRC. https://ncrc.org/lending-discrimination-during-covid-19-black-and-hispanic-women-owned-businesses/
Of the 47 different financial institutions tested in this audit, 19 (40%) institutions had at least one test that showed the control tester was favored.
The discrimination highlighted through the testing reveals how financial institutions stop minority and women entrepreneurs from entering the capital funnel before even applying. Business growth and community development are squashed without access to safe and responsible credit products.
With the high levels of difference in treatment found in two different MSAs on different sides of the country, it is impossible to believe that this is not occurring in other cities. To date, the CFPB has not brought any enforcement actions in the small business lending market. The CFPB must exercise its oversight and enforcement authority under ECOA to ensure that small business owners, especially women and minority owners, have access to safe, affordable, and responsible credit products from financial institutions. The only way to ensure that this occurs is through enforcement actions that the CFPB must start initiating.
1071 Implementation
In September 2020, the CFPB released its outline of policy considerations and questions regarding the Section 1071 small business loan data required by the Dodd-Frank Wall Street Reform and Consumer Protection Act of 2010.[22] Section 1071 of Dodd-Frank requires lending institutions to report demographic information about applications of small business loan, including race and gender of the business owners.[23] Aspects of the proposal, if adopted, will bring much-needed, uniform transparency to the small business credit market by collecting data on application and lending to small business, woman-owned businesses and businesses owned by people of color. The proposals also include disclosure requirements from most types of lenders active in the small business market, including banks, credit unions, online lenders and others, ensuring that lenders cannot exploit loopholes. They also propose collecting data on loan pricing, number of employees, and years in business to ensure fair access for the smallest businesses and start-ups. All of these provisions should be included in a final rule.
A number of approaches considered by the CFPB, however, must not be included in the next step in the regulatory process. Many of these options would make the small business credit market less transparent and would make it difficult to prevent well-documented discrimination. CFPB must require consistent reporting from all types of lenders, including banks, credit unions, online lenders and others. The CFPB should not exempt lenders from the new disclosure requirements based solely on the assets held by the institution. An asset threshold would exclude some small banks and credit unions, which are important small business lenders, and also exclude non-depositories, such as online lenders would be excluded under an assets-only approach. The CFPB should not exempt non-traditional lenders, such as merchant cash advance providers and others.
We urge that the CFPB to move forward with a strong proposed Section 1071 proposed rule in 2021 and issue a final rule in 2022.
Question 6: Sexual Orientation and Gender Identity Discrimination
Should the Supreme Court’s decision in Bostock affect how the Bureau interprets ECOA’s prohibition of discrimination on the basis of sex? If so, in what way(s)?
Discrimination frequently occurs in the lending arena against people who are members of the LGBTQ+ community. Data from the Home Mortgage Disclosure Act (HMDA) shows that from 1990-2015 same-sex applicants were 73.12%[24] more likely to be denied a home loan than different-sex applicants. The addition of rate information in the HMDA disclosures reveals that same-sex applicants pay a higher interest rate and fees than different-sex applicants.[25] Furthermore, analysis of the data showed that female same-sex applicants paid a higher interest rate than male same-sex applicants.
In June, the Supreme Court decided in Bostock v. Clayton County[26] that sex-based discrimination in the employment arena included people who are fired based on sexual orientation or gender identity. The language used in Title VII to describe this protected class is very similar to the language used in ECOA to define the protected class of sex. Bostock is not an expansive reading by the Court; instead, it is a holistic understanding of the term sex. It is not a leap to recognize that if a class of people is discriminated against within the employment arena, they are also being discriminated against within the lending arena and should receive protections to access safe and responsible lending products under ECOA. Furthermore, defense attorneys[27] have already informed the mortgage industry that as more state laws incorporate this robust definition of sex, they should incorporate it into their policies and procedures.
The CFPB needs to release guidance that interprets the protected class of sex within ECOA with the same understanding as the Court does in Bostock.
Question 8: Public Assistance Income
Should the Bureau provide additional clarity under ECOA and/or Regulation B regarding when all or part of the applicant’s income derives from any public assistance program? If so, in what way(s)? For example, should it provide guidance on how to address situations where creditors seek to ascertain the continuance of public assistance benefits in underwriting decisions?
Although ECOA prohibits creditors from discriminating against an applicant because all or part of the applicant’s income is derived from a public assistance program, we are concerned that creditors may be using discretion currently permitted within the law to improperly raise obstacles that make it difficult for applicants to document that income.[28] For example, although a creditor may inquire about the length of time an applicant will continue to receive public assistance, it should not exclude the income if the documentation provided by the government clearly confirms the assistance, but does not contain information about the duration of the assistance.[29] The CFPB issued a bulletin in 2014 to alleviate this concern, but we do not believe the bulletin alone sufficiently addresses it.[30] The bulletin referenced that in connection with its Qualified Mortgage Rule, that a Social Security Administration benefit verification letter that “does not indicate a defined expiration date within three years of loan origination, the credit shall consider the income effective and likely to continue.”[31] The CFPB should amend Regulation B to provide that any category of public assistance that is verified but does not include a specific duration, should be accepted by the creditor and considered for ability to pay. This action would not preclude the creditor from verifying continued public assistance annually to determine continued eligibility for a line of credit. However, any more stringent requirement would be considered discriminatory.
Further, the CFPB should require creditors to value all non-taxable income, including public assistance, fully. The CFPB’s current guidance as part of its amended Appendix Q to the Qualified Mortgage Rule provides that mortgage lenders “may” include the value of tax savings in gross income but does not require they do so.[32] The CFPB notes in Appendix Q that if a mortgage lender does elect to include the additional tax benefit, it must “gross-up” the value of the non-taxable income at the tax rate of the applicant’s last tax year.[33] An applicant who receives $10,000 in non-taxable public assistance should have that income valued at $12,500 if his/her tax rate in the previous year was 25% to reflect the tax savings.[34]The CFPB should go further and not just suggest but require that all creditors gross-up non-taxable income value if they use gross income to determine ability to pay. This change would coordinate with the OCC’s guidance that if a lender uses gross income in its underwriting and has a policy not to gross up non-taxable income to recognize the applicant’s tax benefit, that mortgage lenders’ policy is “likely to be proven discriminatory.”[35]
The CFPB should ensure that persons who receive public assistance have an even playing field with those who do not. Currently, the amount of discretion within ECOA and the additional guidance do not provide consumers receiving public assistance with an equal opportunity.
Question 9: Artificial Intelligence and Machine Learning
Should the Bureau provide more regulatory clarity under the ECOA and/or Regulation B to help facilitate innovation in a way that increases access to credit for consumers and communities in the context of AI/ML without unlawful discrimination? If so, in what ways?
Models need to be both predictive and fair; an active CFPB can make this happen.
Regulators must address the emerging problem of digital redlining. If issues of algorithmic biases are left unaddressed, AI/ML models may repeat past mistakes, where a new generation of discriminatory credit underwriting systems are built from preexisting patterns of inequality and exclusion.[36] Yet confronting the proponents of AI can be difficult; not only is it the case that models are themselves mysterious but often, the modelers who design the systems may overestimate the precision of their outputs. We contend that some skepticism about AI is healthy. AI is only as good as the data set it uses. Moreover, we should be wary of false precision.
First, we must acknowledge that accuracy is something of a mirage. If measured by its ability to make predictions from what it has gleaned from inside a specific data set, the most accurate model may not be the most accurate model when applied to other data sets. We caution regulators to be skeptical of any modeler who claims otherwise. Models can become too local – creating algorithms that improve their ability to see patterns in a baseline data set to the point where their observations no longer hold validity when applied to a different data set. The problem of “overfitting” to local bias is especially confounding when the baseline data set is small or if the loans in a data set vary significantly from real-world norms.
Secondly, there are distinctions in the predictive power of models that are not meaningfully different. If one model is slightly more predictive than another, but the former is much less fair than the latter, shouldn’t our regulatory approach take those differences into account? Despite that, if the first model was only 1/100th of one percent more predictive than the latter, an argument that it was defensible on the grounds of business necessity could be made. We disagree with this perspective. Instead, we believe that the proper approach to regulation of AI is to expect lenders to optimize their models for two goals: accuracy and fairness.
There is a perception that applying fairness criterion to the optimization of a model may undermine its predictive power. Those with this perspective use a zero-sum framework, pitting the interests of profit against the needs for a fair and just financial system. Some people refer to this as the “accuracy – fairness tradeoff,”[37] or as one academic put it, “demanding fairness of models always comes at a cost of reduced accuracy.”[38]
We reject the idea that fairness and accuracy are necessarily opposed to each other. Doing so would itself be a condemnation of AI/ML in general, as it would infer that a complicated model can only improve on one criterion at a time. The table below illustrates this point. It consists of hundreds of points on a “fairness vs. model quality” graph. The chart’s crucial dot is the dark black dot located at the intersection of 100 percent accuracy and 100 percent disparate impact. This dot represents the baseline model. The other dots in the table represent other potential versions of models. A movement to the right on the x-axis indicates a reduction in the disparate impact of a model. For example, an output at the chart’s far-right represents a model whose disparate impact was only 50 percent of the baseline model and thus is fairer. A move upward on the y-axis indicates an improvement in accuracy that improves model quality, or the ability to predict loan performance, including the risk of default. A result above 100 percent is more accurate than the baseline model, but an output below 100 percent is less predictive of loan performance.
Source: BLDS, Inc.
The outputs that are above and to the right of the baseline model illustrate models that were both more accurate and had reduced disparate impacts compared to the baseline model. These are successful outcomes. The CFPB should challenge lenders to improve on their baseline models in just this way.
We would argue that some accommodation should be made to preference gains in fairness – even if the new iteration forces lender to make a modest concession to model quality. The intervention should occur in the most sensitive elements of lending – underwriting, marketing, and collections. Therein, we propose a three-step process for reviewing models:
First, the CFPB should consider using a dataset of its own for testing models of lenders. The CFPB should use a large data set, overcoming the kinds of data overfitting problems that can result when an AI/ML model learns from a dataset that is too narrow. Additionally, the CFPB should update the data frequently. With that dataset, the CFPB should run the lender’s algorithm to examine for fair lending concerns.
Second, the CFPB should establish a threshold for any loan where further examination for potential fair lending concerns is warranted. In cases where an initial review (running the lender’s model with loans from the CFPB’s dataset) suggests disparate impact concerns may exist, the CFPB should go to a third step.
In the third step, the CFPB should run slight alterations of the lender’s originally submitted baseline model. By the process of iterating, the CFPB will soon see if there are ways that can lead to significant reductions in disparate impact with little or minimal losses of model quality.
We describe this third step as the “accommodation ratio.”
Accommodation Ratio
We can calculate the accommodation ratio for a model like this:
(change in the level of disparate impact)
(change in model quality)
An accommodation ratio of 5.0 means that a one percentage point loss of model quality leads to a five-percentage point reduction in disparate impact.
We propose that regulators, at minimum, compel lenders to alter their model when a change from the most predictive approach can create an accommodation ratio of 2.0 or greater and conduct additional research to determine the benefits of setting a lower threshold.
The review process outlined above leads a lender to find a model that more accurately defines risk while reducing disparate impact. This test will create a systematic way of requiring lenders to place fair lending concerns on equal grounds with repayment risk. In any compliance procedure, lenders would have to understand the level of bias in their model, moving their underwriting closer to the previously-mentioned goal of explainability. By forcing a second criterion – not just predictive but also fair – modelers will have to go back to the “modeling drawing board.” A good AI/ML model, when allowed to iterate, should be able to find the right mix of variables that lead to improvements in both goals.
Methods for Reviewing Algorithms
By design, AI/ML tests the limits of human understanding. The recommendations we have made for adding explainability to modeling in Question 10 also apply to issues raised here. For a regulator to review hundreds of models on an ongoing basis requires a framework for review. We have recommended that the CFPB publish guidance to lenders on our proposed accommodation ratio. Additionally, other elements in the guidance could provide clarity for lenders when they develop their baseline models.
Quantitative monotonic constraints: The advantage – but also the inherent risk – of AI/ML is its ability to incorporate thousands of variables into models that can identify patterns that would otherwise be missed by human judgment. Nonetheless, models can only make conclusions based on the data they are given. If the set of loans used in analysis differs from the lending in the broader marketplace, it can lead the algorithm to make conclusions that lack real-world validity.
Monotonic constraints safeguard against nonsensical conclusions. A monotonic constraint controls the direction that any input can have on the outcome. For example, if a small dataset included many refinance mortgage loans that fell into foreclosure, even though the loans had loan-to-value ratios of well below 50 percent, the model might associate higher risk rates with lower LTV ratios. Such a system would penalize borrowers who made higher down payments. A model builder should proactively intervene to prevent this kind of overfitting error by applying a monotonic constraint that said that low LTV ratios should not increase the predicted default rate. In essence, a monotonic constraint is a tool to safeguard against errors that are the product of unrepresentative data sets.
Notably, monotonic constraints are easy to implement, increase explainability, lead to models that are more readily understood by loan applicants, and improve model quality.
Test data sets for real-world validity: As mentioned above, problems can occur when the wrong data set is used for AI/ML modeling. The CFPB should, as a part of testing models, examine how base data sets used by lenders may lead to biased outcomes from the models used by those lenders. Risks of overfitting to biased models may be substantial when data sets include only a small number of borrowers from protected classes.
Cultural bias: To optimize the benefit of human review of AI/ML, we must start by guaranteeing that we have the right human judgment. The CFPB should provide guidance for lenders that underscores the need to apply culturally-competent values in the management of AI/ML systems. Lenders should incorporate processes in their AI training to guard against sociological bias as a part of fair lending compliance.[39]
Stress-testing: Given the relative infancy of AI in the financial sector, some models may not be predictive during times of crisis, such as the current pandemic. Some AI/ML models failed during the 2009 financial crisis when the outside world produced unexpected conditions. Traditional regression-based underwriting systems (FICO, for example) have the advantage of being well-understood by banking professionals, but the same cannot be said of AI/ML. Moreover, the risk grows when they do not understand the logic of their models. The risk that lenders may not fully understand their AI/ML models and as a result restrict lending during times of crisis underscore why the CFPB should apply stress tests to AI/ML models.
Lack of technological capacity to identify fair lending violations does not justify non-compliance. Underwriting through AI/ML is difficult – far more than using traditional relationship lending or even using regression models – but difficulty is not a reason to shed accountability. The CFPB should not permit lenders to ignore fair lending concerns associated with AI/ML because of a lack of technological capability.
Question 10: ECOA Adverse Action Notices
Should the Bureau provide any additional guidance under ECOA and/or Regulation B related to when adverse action has been taken by a creditor, requiring a notification that includes a statement of specific reasons for adverse action? If so, in what ways?
Adverse action notices answer a vital need – explaining an unfavorable credit decision – but the migration by financial institutions to underwriting with artificial intelligence (AI) and machine learning (ML) techniques poses new challenges to making notices that are both consistent with a model’s conclusions and explainable in terms that applicants can understand. The chasm between the complexity of AI/ML models and non-technical audiences’ ability creates a substantial barrier to meeting the needs that inspired the adverse action notice.[40]
Generating clear and explainable action notices can improve fair lending policy and model review practices. Lenders must start with “the end in mind” to create easily-interpretable notices that flow from building models that are comprised of variables that humans can understand.
A meaningful adverse action notice meets four criteria: it is accurate, explainable, interpretable, and actionable.
- Accurate: The notice conveys reason codes that are faithful to the actual reasons behind the decision. It is not satisfactory to provide proximate causes or ones that collapse multiple variables from the model into one summary variable, as such a method is opaque.
- Explainable: A lender should understand how its model works. When speaking to regulators and advocates, lenders should not be allowed to claim a defense that a model is too complicated to determine if it includes discriminatory constructs.
- Interpretable: When creating its adverse action notice, the lender should explain the model so that the end-user (consumer or business owner) can interpret it. If a lender collapses multiple variables into one reason code, consumers may make wrong assumptions about which behaviors led to an adverse decision.
- Actionable: shows an applicant how with different behavior in the future, they could receive a favorable outcome next time. Modelers have to start with variables that leave humans with the agency to improve. Actionability goes back to the beginning of the modeling ideation. If lenders use inputs that could leave borrowers without a reasonable way to change how they are evaluated, it thwarts actionability. Similarly, both explainability and interpretability have to be achieved if borrowers can have the agency to contest their data’s accuracy.
Accuracy
Compared to the well-established structure of a traditional FICO model,[41] AI/ML models can be more complicated, tend to change frequently, and weigh a factor differently from one application to the next. Each of these creates challenges for translation. With so many variables, limiting an adverse action notice to four or five reasons may be too simplistic. When models evolve continuously, it is hard to create coding systems.
Traditionally, financial institutions (FIs) have called on three primary methods for determining which variables were the most significant factors in an adverse decision. Some FIs have reviewed the inputs in their models to ensure no prima facie bias or grounds to believe that a data point input could contain proxy bias. Some have chosen the “drop one” method, where a single variable is dropped from a model to see which absence has the most significant impact on the prediction. While the latter approach requires more rigor and may be more statistically sound, neither is sufficiently robust method for selecting an adverse action reason with AI. Only a subset of FIs uses the most scientifically robust testing tools, such as Shapley values, which describe the contribution made by each variable to an outcome in a specific lending decision to make statistically valid assessments of an input’s contribution when building their adverse action notice systems.[42]
Explainability and interpretability
While explaining underwriting techniques constructed with linear and logistic regression models are reasonably straightforward, the same is often not valid for models that deploy AI/ML and ML. The contribution of a variable on the final result (expressed as a Shapley value) may differ by applicant and by FI. It is important for each FI to be able to demonstrate that the method it uses to explain its decision is not just clearly described, but that it can be understood by the applicant.
Actionable
FIs should publish adverse action notices that contain information that can help an applicant improve their creditworthiness so that the applicant could receive a more favorable outcome in the future. The leading regression-based underwriting models use actionable variables, but many AI-based models include variables that are not actionable. An explainable model can express the same factors in easily interpretable terms that led to the model’s decision. If a well-explained adverse action notice reveals that an AI/ML model used factors that leave the applicant without the ability to improve her credit score, then the adverse action notice was not effective.
Modelers can create explainable models without compromising on the overall predictive power of the algorithm. That point was illustrated recently during a several-day competition among data scientists to produce a model from data provided by a credit bureau. Most of the modelers resorted to making “black box” models, but one team chose to use only explainable variables. The explainable team found that they lost only 1 percent in overall accuracy – a difference that was within the overall margin of error.[43] With that de minimis difference, however, the model’s conclusions were no longer shrouded in mystery. That matters if we want to make sure that AA notices remain actionable for consumers.
Adverse Action notices only have room to list several reason codes. Unfortunately, when variables are collapsed to fit within those constraints, it compromises the effectiveness of the notices.
The CFPB should address problems that occur when well-intentioned modelers attempt to explain their models within the constraints of the standard AA notice. The table below illustrates the issue:
| Feature input | AA Notice Expression |
| Cash flows for wages or government benefits | Monthly Income |
| Cash flows from P2P systems | |
| Cash flows from reimbursements | |
| Mortgage payment | Monthly spending |
| Auto loan payment | |
| Student loan payment | |
| Transfers to 529 plan | |
| Other spending | |
| Credit card balance – department store card | Credit card utilization |
| Credit card – a personal card used for business travel | |
| Number of delinquencies 30+ days | Number of delinquencies |
| Number of delinquencies 60+ days |
This table is meant for illustration purposes only.
This table highlights a structural problem with adverse action notices. Most notices include only four or five reason codes. In this case, by reducing the number of action codes to four, the modelers have had to collapse variables in ways that reduced the accuracy of the explanation. In some cases, the summarization is a difference without a distinction, as expressed here by collapsing two delinquency variables into one. However, others reveal problems.
Not all cash flows should qualify as income, but applicants cannot be sure how their data was seen by the lender when the adverse action notice collapses all income-related variables. Some consumers may receive cash flows that look like income from account-to-account transfers. Indeed, a subset of P2P transfers could represent informal sources of earned income. The problem is that the consumer does not know how the model interpreted those transfers. While a model may know which sources of positive cash flows are income, a doubt could remain in consumers’ minds. Such a situation – where the consumer does not see the basis of the credit assessment – is a suboptimal outcome.
With “monthly spending,” the modeler has been forced to collapse recurring payments with divergent meanings. It is possibly a sign of a problem when recurring payments for housing and transportation costs are too high, but decidedly not when an applicant has been transferring money to a different financial institution to prepare for college tuition expenses.
With credit card utilization, there are meaningful differences between outstanding balances on a store credit card and on balances that exist because the applicant is waiting for reimbursement.
In both the “monthly spending” and “credit card utilization” examples, it is likely that the original model embedded the distinctions mentioned above in the model. Still, that is little consolation, as the message to the consumer is muddled. The consumer may never conceive of the need to save less to improve his credit profile. Moreover, it’s not clear in the adverse action notice if the modeler used reimbursable business-related travel expenses.
The CFPB should provide guidance on how to create notices that overcome the problem of collapsing multiple related variables into a single reason code.
A better approach
With the advent of digital technology, many ways already exist to improve an adverse action notice’s interpretability and actionability. We should start by acknowledging the value of digital over paper. Paper is unnecessarily one-way; it lacks a means for applicants to react with the notice in ways that could answer unresolved questions. As well, as it requires printing and then delivery through the mail, it arrives well after the application was denied, even if the decision was made in only a few minutes. There is no way for a consumer to respond to the information. Moreover, the use of paper imposes limits on the amount of content that can exist in a notice.
With a digital alternative, adverse action notices could become two-way and more expansive. In the case listed in the previous table, where the utilization of too high a share of available credit utilization harmed an applicant’s score, a digital tool could let the consumer expand the reason code in an accordion fashion. Such an approach would use a Shapley value, and the accordion could reveal that the model discounted the contribution of the business-related charges to the overall decision. Similarly, if the applicant expanded the accordion to understand better how the monthly spending factors that influenced the decision, the expression could show how each expenditure influenced the model output – including to show that transfers to a 529 were not negative. Ultimately, such a system enhances the ability of the applicant to take actions to improve his credit score.
ECOA and the Community Reinvestment Act
The CFPB should work on an interagency basis with the federal bank regulatory agencies to apply its updated ECOA guidance to fair lending reviews that are concurrent with CRA exams. Findings of widespread and severe discrimination can reduce CRA ratings awarded to banks. The CFPB and the banking agencies should issue guidance that rigorous fair lending testing and protections in AI is required and will inform a bank’s CRA rating.
Conclusion
The CFPB needs to provide more guidance on the nuances of ECOA to ensure the delicate balance between lending and eradicating discrimination. ECOA is a critical tool to elevate consumers and secure access to safe and responsible credit products. With a successful implementation of changes, the negative impacts of the current economic crisis could be reduced, and ECOA can be strengthened for years to come.
If you have any questions, please contact me or Tom Feltner, Director of Policy, at (202) 524-4889 or tfeltner@ncrc.org.
Sincerely,
Jesse Van Tol, Chief Executive Officer
[1] NCRC did not respond to question 7
[2] HUD’s Implementation of the Fair Housing Act’s Disparate Impact Standard, 85 Fed. Reg. 60,288, 60,332 (Sept. 24, 2020) (codified at 24 C.F.R. pt. 100.500).
[3] Implementation of the Fair Housing Act’s Discriminatory Effects Standard, 78 Fed. Reg. 11,459, 11,482 (Feb. 15, 2013) (codified at 24 C.F.R. pt. 100.500).
[4] HUD’s Implementation of the Fair Housing Act’s Disparate Impact Standard, 84 Fed. Reg. 42,854, 42,862 (Aug. 19, 2019).
[5] HUD’s Implementation of the Fair Housing Act’s Disparate Impact Standard, 85 Fed. Reg. at 60,288.
[6] See, e.g. OCC Bulletin, 2013-29, “Third Party Relationships: Risk Management Compliance (October 30, 2013).
[7] Consumer Financial Protection Bureau. Nov 2017. “Spotlight on serving limited English proficient consumers: Language access in the consumer financial marketplace.” https://www.consumerfinance.gov/data-research/research-reports/spotlight-serving-limited-english-proficient-consumers/
[8] See Glossary of English-Financial Terms (October, 2018); Glossary of English-Chinese Financial Terms (February, 2019)
[9] Final Language Access Plan for the Consumer Financial Protection Bureau, Federal Register, Vol. 82, No. 220 at 60842 (November 16, 2017).
[10] 15 U.S.C. 1691(c)(3)
[11] 12 CFR section 1002.8
[12] See, e.g. 12 C.F.R. Part 1002, Supplement 1, Sections 8(c) and (d); CFPB, Supervisory highlights, Issue 12, Sumer 2016, Section 2.52; OCC, Community Developments, Summer 2001, p. 22.
[13] 12 C.F.R. 1002.8(3)(i) and (ii).
[14] It is the CFPB’s “responsibility to promulgate a regulation that is consistent with Federal and state law. 12 C.F.R. Part 1002, Supplement 1, section 8(a), comment 2.
[15] Compliance Bulletin and Policy Guidance; 2016-02 https://www.federalregister.gov/documents/2016/10/26/2016-25856/compliance-bulletin-and-policy-guidance-2016-02-service-providers
[16] Consumer Financial Protection Bureau (2017). Spotlight on serving limited English proficient consumers: Language access in the consumer financial marketplace. Retrieved at https://files.consumerfinance.gov/f/documents/cfpb_spotlight-serving-lep-consumers_112017.pdf
[17] Long, H. May 12, 2020. “Small business used to define America’s economy. The pandemic could change that forever.” Washington Post. https://www.washingtonpost.com/business/2020/05/12/small-business-used-define-americas-economy-pandemic-could-end-that-forever/
[18] Liu, Sifan and Joseph Parilla. (2020, September 17). New data shows small businesses in communities of color had unequal access to federal COVID-19 relief. Brookings Institution. Retrieved at https://www.brookings.edu/research/new-data-shows-small-businesses-in-communities-of-color-had-unequal-access-to-federal-covid-19-relief/
[19]Federal Stimulus Survey Findings. (2020, May 13) Unidos US. https://theblackresponse.org/wp-content/uploads/2020/05/COC-UnidosUS-Abbreviated-Deck-F05.13.20.pdf
[20] Liu, Sifan and Joseph Parilla. (2020, September 17). New data shows small businesses in communities of color had unequal access to federal COVID-19 relief. Brookings Institution. Retrieved at https://www.brookings.edu/research/new-data-shows-small-businesses-in-communities-of-color-had-unequal-access-to-federal-covid-19-relief/
[21] Richardson, Jason and Jad Elebi. (2020, July 15) Government Data on PPP Loans is Mostly Worthless. But It’s Not Too Late to Fix It. National Community Reinvestment Coalition. Retrieved at https://ncrc.org/government-data-on-ppp-loans-is-mostly-worthless-but-its-not-too-late-to-fix-it/
21 Consumer Financial Protection Bureau. (2020, September 15). Consumer Financial Protection Bureau Releases Outline of Proposals Under Consideration to Implement Small Business Lending Data Collection Requirements. Retrieved at
22 Consumer Financial Protection Bureau. Small business lending data collection rulemaking: We are now in the process of writing regulations to implement section 1071 of the Dodd-Frank Act. Retrieved at https://www.consumerfinance.gov/1071-rule/
[24] Sun, H., & Gao, L. (2019). Lending practices to same-sex borrowers. Proceedings of the National Academy of Sciences, 116(19), 9293-9302.(App. B)
[25] Richardson, J & Kali, K. June 22, 2020. “Same-Sex Couples and Mortgage Lending” NCRC. https://ncrc.org/same-sex-couples-and-mortgage-lending/
[26] Bostock v. Clayton County. Vol. No. 17–1618 (2019) https://www.supremecourt.gov/opinions/19pdf/17-1618_hfci.pdf
[27] Willis, C. & Culhane Jr., J. (2020, June 19) . “SCOTUS decision on Title VII sexual orientation discrimination has significant implications for credit arena.” Consumer Finance Monitor. Retrieved at https://www.consumerfinancemonitor.com/2020/06/19/scotus-decision-on-title-vii-sexual-orientation-discrimination-has-significant-implications-for-credit-arena/
[28] See15 U.S.C. section 1691(a)(2) and (b)(2); 12 C.F.R. part 1002, Supp. I, section 1002.6 paragraph 6(b)(2)-6(i).
[29] Id.
[30] See CFPB Bulletin 2014-03, Social Security Disability Income Verification (November 18, 2014). Retrieved at https://files.consumerfinance.gov/f/201411_cfpb_bulletin_disability-income.pdf
[31] 12 CFR pt. 1026, App. Q, Section I.B.11 & Note i.
[32] 12 CFR pt. 1026, App. Q, Section E.2.
[33] Id.
[34] $10,000 x 125% = $12,500
[35] OCC Comptroller’s Handbook, Fair Lending, January 2013 at 163. https://www.occ.treas.gov/publications-and-resources/publications/comptrollers-handbook/files/fair-lending/pub-ch-fair-lending.pdf
[36] Barocas, Solon and Andrew Selbst (2016). “Big data’s disparate impact.” 104 California Law Review 671. Retrieved at: https://ssrn.com/abstract=2477899
[37] Wick, Michael and Jean-Baptiste Tristan. (2020). Unlocking Fairness: A Tradeoff Revisited. Advances in Neural Information Process Systems, 32 Retrieved at https://papers.nips.cc/paper/2019/hash/373e4c5d8edfa8b74fd4b6791d0cf6dc-Abstract.html
[38] Berk, Richard, Hoda Heidari, Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, Seth Neel, and Aaron Roth. (2017, June 9). “A Convex Framework for Fair Regression.” Retrieved at https://arxiv.org/pdf/1706.02409.pdf
[39] Leong, Brenda (2020, June 12). Ten Questions on AI Risk: Gauging the Liabilities of Artificial Intelligence Within Your Organization.” Retrieved at https://fpf.org/2020/06/12/ten-questions-on-ai-risk/
[40] BLDS, LLC., Discover Financial Services, and H20.ai. “Machine Learning: Considerations for Expanding Access to Credit Fairly and Transparently.” https://www.h2o.ai/resources/white-paper/machine-learning-considerations-for-fairly-and-transparently-expanding-access-to-credit/
[41] FICO Scores are calculated using data categories from an applicant’s credit report. While a FICO score is the product of many inputs, the contributions are given a uniform weighting within five categories: payment history (35%), amounts owed (30%), length of credit history (15%), new credit (10%) and credit mix (10%). A coefficient is a weighting.
[42] Rodríguez-Pérez, R., Bajorath, J. Interpretation of machine learning models using Shapley values: application to compound potency and multi-target activity predictions. Journal of Computer-Aided Molecular Design. Vol 34, 1013–1026 (2020). https://doi.org/10.1007/s10822-020-00314-0
[43] Rudin, Cynthia and Joanna Radin. 2019. “Why Are We Using Black Box Models in AI When We Don’t Need To? A Lesson from An Explainable AI Competition.” Harvard Data Science Review. Retrieved at https://hdsr.mitpress.mit.edu/pub/f9kuryi8/release/5
