Rapid evolution in artificial intelligence (AI)
applications, as well as improvements in computing power and the increasing
availability of data, have led to significant growth in AI across most
industries. The key developments in AI over the past few years have been driven
by machine learning which, in turn, is fuelled by data. As more and more data
is being gathered, so AI enables more sophisticated analysis of large data
volumes. As the importance of data rises, so do the associated legal
issues.
In some cases businesses are free to use the data
they hold for whatever purpose they want, including developing AI
algorithms. However, there are many instances where they are not free to
use the data. For example, if personal data is used to develop, train or test
AI algorithms, that processing will need to be fair and lawful and comply with
data protection laws. In addition, if the data relates to a third party,
it might be confidential or provided under a limited licence. Also, if third
parties will have access to the data, that may complicate the data protection
and confidentiality issues.
It is of the utmost importance that AI is used
responsibly. On a global scale, an important step to strengthen trust in AI has
been the development of principles for the responsible development and
deployment of AI, including accountability and transparency. One important
aspect of ensuring responsible deployment of AI is legal and regulatory
compliance.
From a compliance perspective, businesses in South
Africa will firstly need to ensure that their AI system is compliant with the
Protection of Personal Information Act, 2013 (POPIA). Almost all of the remaining provisions of POPIA came into
effect on 1 July 2020. Various aspects of POPIA must be considered
when creating an AI system.
Of key importance for AI systems, given their inherent
problem-solving ability, is section 71(1) of POPIA, which governs automated
decision-making. This section protects data subjects from being subjected to a
decision which is based solely on automated decision-making, which results in
legal consequences for the data subject and the data subject being profiled.
For instance, an AI system would have the ability to
profile customers seeking a bank loan, and determine their creditworthiness
based on previous loan repayments, income, indebtedness etc. Section
71(1) prohibits the bank from making a decision to grant or reject the loan
application based solely on the profile created by the AI system. In this
particular example, however, the bank which is receiving the customer’s loan
application would need to determine whether it can rely on one of the
exceptions to the prohibition on automated decision-making, which are set out
in section 71(2) of POPIA.
Businesses implementing an AI system should also be
mindful of section 57(1)(a) of POPIA, which requires a responsible party to
obtain prior authorisation from the Information Regulator if it intends to
process any unique identifiers of data subjects (i) for another purpose than
intended at collection, and (ii) with the aim of linking the information with
information processed by other responsible parties. In this instance,
“unique identifier” can be any identifier that uniquely identifies a
data subject in relation to the responsible party such as, for example, an
identity number or employee number.
Section 57(1) will be relevant when, for example, an
AI system deployed by Business A intends to combine the identity number of an
employee with data collected by Business B to determine whether the employee is
more susceptible to a certain work-related risk based on his or her age.
In this instance, Business A would have to approach the Information Regulator
before it could utilise the AI system. The responsible party must
consider not only what information will be processed by the AI system but also how
the AI system will use it, to ensure that all data protection compliance
requirements have been met.
In many instances, AI systems learn and become more
intuitive by acquiring vast quantities of data. However, as mentioned above,
organisations must proceed with caution when the data inadvertently contains
personal information. Before feeding the data into the system, the organisation
should consider whether the information can be input in de-identified form,
which would exclude it from the application of POPIA. If the information
cannot be de-identified, it would be incumbent on the responsible party to
ensure that data subjects are aware that their personal information is being
used to test an AI system. If the data subjects originally provided their
personal information to the organisation for the purposes of procuring a
product or service, for example, then the use of the personal information for
AI testing must be compatible with that purpose. If not, before the
organisation can use the data subjects’ personal information for AI testing
purposes, it would need to obtain their consent.
While the deployment of AI systems creates great
opportunities for organisations, it is important for them to understand the
laws that apply to the data being input into the system to ensure that use of
data in the AI system is not in breach of any laws.
