人工智能也有偏见与歧视，如何解决成为行业难题

IBM和Salesforce是人工智能工具软件领域的两家领军企业，这两家公司的高管在近日指出，对于那些希望采用人工智能软件的公司来说，偏见仍然是一个值得担忧的根本性问题。

让很多企业越来越担心的是，用于训练人工智能系统的数据中所隐藏的偏见，有可能会导致系统生成的结果对某些应该受到保护的群体（比如女性和少数族裔）做出不公平的结论，甚至造成歧视或涉嫌违法。

比如有人发现，一些人脸识别系统在识别深肤色和浅肤色人脸时，精确度往往不高，原因是用于训练该系统的深肤色人脸数据远远不足。最臭名昭著的一个例子是，美国部分州的司法部门使用了一套人脸识别系统，用来决定是否应该批准犯罪嫌疑人保释或假释。然而在犯罪记录相似的前提下，系统却认为黑人嫌犯比白人嫌犯有更高的再次犯罪风险。

软件公司Salesforce的首席科学家理查德·佐赫尔在科罗拉多州阿斯彭市举办的《财富》头脑风暴科技大会上指出：“偏见将成为未来人工智能领域里的一个根本性问题。”

在这次《财富》头脑风暴科技大会上，IBM公司的研究总监达里奥·吉尔也表达了同样的担忧：“我们需要在人工智能工程上采取坚实手段，以防止人工智能出现毫无根据的偏见。”

吉尔表示，IBM正在加大相关技术的研发力度，为企业提供所谓“数据志”功能。这种技术能够记录系统是使用哪些数据决策的，这些数据又是如何生成的、何时被使用的，以及它是如何用于进行推荐或预测的。

吉尔说，这种人工智能的审查跟踪机制对于问责十分重要，毕竟责任终究是要由人来承担的。他表示：“我们必须把责任落实到开发这个软件的人身上，我们要知道他们的目的是什么、意图是什么。创建和使用这个软件的机构必须要承担责任。”

吉尔和佐赫尔都表示，消除人工智能的偏见并不是一件容易的事情，特别是机器学习系统非常擅于发现数据集里各个变量的相关性。因此，虽然我们可以告诉这些软件在进行相关决策时（比如提供征信方面的建议）不考虑种族因素，但系统仍然会考虑到一个人的住址或邮编等变量。佐赫尔指出，至少在美国，像地址、邮编等信息，实际上还是有可能与族裔群体高度相关的。

吉尔还表示，着眼这一问题，IBM已经开发了一些相关软件，比如它的AI Fairness 360工具包，可以帮助企业自动在数据中发现类似隐藏的相关性问题。

不过，佐赫尔也指出，发现这种相关性是一回事，但从很多方面看，知道究竟应该怎样解决它，则是一个困难得多的问题。

佐赫尔表示，在某些情况下，只将一种产品推荐给女性是没有问题的——比如吸奶器。而在其他情况下，如果系统在进行推荐时出现了类似的性别歧视，则可能涉嫌违法。Salesforce等公司生产的一些通用型人工智能工具几乎各行各业都可以使用，因此他们面临的困难也尤为特殊。

吉尔和佐赫尔都表示，正因为如此，很多企业才选择用自己的数据来训练人工智能系统，而不是使用已经使用预先训练好的软件包来执行聊天机器人或自动图像标记程序的训练任务。吉尔指出，构建自己的人工智能程序，让企业掌握了更大的控制权，同时也更有可能检测出隐藏的偏见。

佐赫尔和吉尔还表示，人工智能的优点之一，就是它能够帮助企业发现其实际业务中现有的偏见因素。比如，它可以发现哪些管理者不愿意提拔女性员工，哪些金融机构不愿意向少数族裔发放信贷等等。佐赫尔表示：“人工智能有时就像我们面前的一面镜子，它会告诉你，这就是你一直以来在做的事情。”

佐赫尔认为，在构建人工智能系统的人自身变得更加多元化之前，有些类型的偏见是不太可能被彻底消除的。目前，很多从事人工智能软件开发的计算机工程师都是白人，而且当前开发的很多人工智能软件都只反映了城市富裕人口的需求。他还表示，这也是Salesforce公司何以支持非洲深度学习大会（Deep Learning Indaba）等项目的原因之一。非洲深度学习大会也是非洲地区人工智能研究人员的一次盛会。（财富中文网）

译者：朴成奎

Bias will continue to be a fundamental concern for businesses hoping to adopt artificial intelligence software, according to senior executives from IBM and Salesforce, two of the leading companies selling such A.I.-enabled tools.

Companies have become increasingly wary that hidden biases in the data used to train A.I. systems may result in outcomes that unfairly—and in some cases illegally—discriminate against protected groups, such as women and minorities.

For instance, some facial recognition systems have been found to be less accurate at differentiating between dark-skinned faces as opposed to lighter-skinned ones, because the data used to train such systems contained far fewer examples of dark-skinned people. In one of the most notorious examples, a system used by some state judicial systems to help decide whether to grant bail or parole was more likely to rate black prisoners as having a higher risk of re-offending than white prisoners with similar criminal records.

“Bias is going to be one of the fundamental issues of A.I. in the future,” Richard Socher, the chief scientist at software company Salesforce, said. Socher was speaking at Fortune’s Brainstorm Tech conference in Aspen, Colo.

Dario Gil, director of research at IBM, also speaking at Brainstorm Tech, echoed Socher’s concerns. “We need robust A.I. engineering to protect against unwarranted A.I. bias,” he said.

At IBM, Gil said, the company was increasingly looking at techniques to provide businesses with a “data lineage” that would record what data a system used to make a decision, how that data was generated and how and when it was used to make a recommendation or prediction.

Gil said this kind of A.I. audit trail was essential for ensuring accountability, something he said must always reside with human-beings. “We have to put responsibility back to who is creating this software and what is their purpose and what is their intent,” he said. “The accountability has to rest with the institutions creating and using this software.”

Both Gil and Socher said that eliminating A.I. bias was not an easy problem to solve, especially because machine learning systems were so good at finding correlations between variables in data sets. So, while it was possible to tell such software to disregard race when making, for example, credit recommendations, the system might still use a person’s address or zip code. In the U.S., at least, that information can also be highly correlated with race, Socher said.

Gil said that IBM has been developing software—such as its AI Fairness 360 toolkit—that can help businesses automatically discover such hidden correlations in their data.

But, Socher said, discovering such correlations is one thing. Knowing exactly what to do about them is, in many ways, a much harder problem.

Socher said that in some cases, such as marketing breast pumps, it might be alright to only recommend a product to women. Meanwhile, in other contexts, the same sort of gender discrimination in recommendations would be illegal. For a company like Salesforce that is trying to build A.I. tools that are general enough that companies from any industry can use them for almost any use case, this presents a particular dilemma, he said.

This is one reason, both Gil and Socher said, many businesses are choosing to train A.I. systems from their own data rather than using pre-trained software packages for tasks chatbots or automated image-tagging. Building their own A.I., Gil said, gave businesses more control and more chances to detect hidden biases.

Both Socher and Gil said that one of the great things about A.I. is that it can help companies uncover existing bias in their business practices. For instance, it can managers who don’t promote women or financial institutions that don't extend credit equally to minorities. “A.I. sometimes puts a mirror in front of our faces and says this is what you have been doing all the time,” Socher said.

He also said that certain types of bias were unlikely to be resolved until the people building A.I. systems were themselves more diverse. At the moment, he said, too many of the computer scientists creating A.I. software are white men. He also said too many of the A.I. applications developed so far reflect the concerns of affluent urbanites. He said this is one reason Salesforce has been supporting projects like the Deep Learning Indaba, a conference designed to bring together A.I. researchers from across Africa.