AutofillService Class

Definition

[Android.Runtime.Register("android/service/autofill/AutofillService", ApiSince=26, DoNotGenerateAcw=true)]
public abstract class AutofillService : Android.App.Service

[<Android.Runtime.Register("android/service/autofill/AutofillService", ApiSince=26, DoNotGenerateAcw=true)>]
type AutofillService = class
    inherit Service

Inheritance

Object

Object

Context

ContextWrapper

Service

AutofillService

Attributes

RegisterAttribute

Remarks

An AutofillService is a service used to automatically fill the contents of the screen on behalf of a given user - for more information about autofill, read Autofill Framework.

An AutofillService is only bound to the Android System for autofill purposes if: <ol> <li>It requires the android.permission.BIND_AUTOFILL_SERVICE permission in its manifest. <li>The user explicitly enables it using Android Settings (the Settings#ACTION_REQUEST_SET_AUTOFILL_SERVICE intent can be used to launch such Settings screen). </ol>

"BasicUsage"><h3>Basic usage</h3>

The basic autofill process is defined by the workflow below: <ol> <li>User focus an editable View. <li>View calls AutofillManager#notifyViewEntered(android.view.View). <li>A ViewStructure representing all views in the screen is created. <li>The Android System binds to the service and calls #onConnected(). <li>The service receives the view structure through the #onFillRequest(FillRequest, CancellationSignal, FillCallback). <li>The service replies through FillCallback#onSuccess(FillResponse). <li>The Android System calls #onDisconnected() and unbinds from the AutofillService. <li>The Android System displays an autofill UI with the options sent by the service. <li>The user picks an option. <li>The proper views are autofilled. </ol>

This workflow was designed to minimize the time the Android System is bound to the service; for each call, it: binds to service, waits for the reply, and unbinds right away. Furthermore, those calls are considered stateless: if the service needs to keep state between calls, it must do its own state management (keeping in mind that the service's process might be killed by the Android System when unbound; for example, if the device is running low in memory).

Typically, the #onFillRequest(FillRequest, CancellationSignal, FillCallback) will: <ol> <li>Parse the view structure looking for autofillable views (for example, using android.app.assist.AssistStructure.ViewNode#getAutofillHints(). <li>Match the autofillable views with the user's data. <li>Create a Dataset for each set of user's data that match those fields. <li>Fill the dataset(s) with the proper AutofillIds and AutofillValues. <li>Add the dataset(s) to the FillResponse passed to FillCallback#onSuccess(FillResponse). </ol>

For example, for a login screen with username and password views where the user only has one account in the service, the response could be:

new FillResponse.Builder()
                .addDataset(new Dataset.Builder()
                    .setValue(id1, AutofillValue.forText("homer"), createPresentation("homer"))
                    .setValue(id2, AutofillValue.forText("D'OH!"), createPresentation("password for homer"))
                    .build())
                .build();

But if the user had 2 accounts instead, the response could be:

new FillResponse.Builder()
                .addDataset(new Dataset.Builder()
                    .setValue(id1, AutofillValue.forText("homer"), createPresentation("homer"))
                    .setValue(id2, AutofillValue.forText("D'OH!"), createPresentation("password for homer"))
                    .build())
                .addDataset(new Dataset.Builder()
                    .setValue(id1, AutofillValue.forText("flanders"), createPresentation("flanders"))
                    .setValue(id2, AutofillValue.forText("OkelyDokelyDo"), createPresentation("password for flanders"))
                    .build())
                .build();

If the service does not find any autofillable view in the view structure, it should pass null to FillCallback#onSuccess(FillResponse); if the service encountered an error processing the request, it should call FillCallback#onFailure(CharSequence). For performance reasons, it's paramount that the service calls either FillCallback#onSuccess(FillResponse) or FillCallback#onFailure(CharSequence) for each #onFillRequest(FillRequest, CancellationSignal, FillCallback) received - if it doesn't, the request will eventually time out and be discarded by the Android System.

"SavingUserData"><h3>Saving user data</h3>

If the service is also interested on saving the data filled by the user, it must set a SaveInfo object in the FillResponse. See SaveInfo for more details and examples.

"UserAuthentication"><h3>User authentication</h3>

The service can provide an extra degree of security by requiring the user to authenticate before an app can be autofilled. The authentication is typically required in 2 scenarios: <ul> <li>To unlock the user data (for example, using a main password or fingerprint authentication) - see FillResponse.Builder#setAuthentication(AutofillId[], android.content.IntentSender, android.widget.RemoteViews). <li>To unlock a specific dataset (for example, by providing a CVC for a credit card) - see Dataset.Builder#setAuthentication(android.content.IntentSender). </ul>

When using authentication, it is recommended to encrypt only the sensitive data and leave labels unencrypted, so they can be used on presentation views. For example, if the user has a home and a work address, the Home and Work labels should be stored unencrypted (since they don't have any sensitive data) while the address data per se could be stored in an encrypted storage. Then when the user chooses the Home dataset, the platform starts the authentication flow, and the service can decrypt the sensitive data.

The authentication mechanism can also be used in scenarios where the service needs multiple steps to determine the datasets that can fill a screen. For example, when autofilling a financial app where the user has accounts for multiple banks, the workflow could be:

<ol> <li>The first FillResponse contains datasets with the credentials for the financial app, plus a "fake" dataset whose presentation says "Tap here for banking apps credentials". <li>When the user selects the fake dataset, the service displays a dialog with available banking apps. <li>When the user select a banking app, the service replies with a new FillResponse containing the datasets for that bank. </ol>

Another example of multiple-steps dataset selection is when the service stores the user credentials in "vaults": the first response would contain fake datasets with the vault names, and the subsequent response would contain the app credentials stored in that vault.

"DataPartioning"><h3>Data partitioning</h3>

The autofillable views in a screen should be grouped in logical groups called "partitions". Typical partitions are: <ul> <li>Credentials (username/email address, password). <li>Address (street, city, state, zip code, etc). <li>Payment info (credit card number, expiration date, and verification code). </ul>

For security reasons, when a screen has more than one partition, it's paramount that the contents of a dataset do not spawn multiple partitions, specially when one of the partitions contains data that is not specific to the application being autofilled. For example, a dataset should not contain fields for username, password, and credit card information. The reason for this rule is that a malicious app could draft a view structure where the credit card fields are not visible, so when the user selects a dataset from the username UI, the credit card info is released to the application without the user knowledge. Similarly, it's recommended to always protect a dataset that contains sensitive information by requiring dataset authentication (see Dataset.Builder#setAuthentication(android.content.IntentSender)), and to include info about the "primary" field of the partition in the custom presentation for "secondary" fields&mdash;that would prevent a malicious app from getting the "primary" fields without the user realizing they're being released (for example, a malicious app could have fields for a credit card number, verification code, and expiration date crafted in a way that just the latter is visible; by explicitly indicating the expiration date is related to a given credit card number, the service would be providing a visual clue for the users to check what would be released upon selecting that field).

When the service detects that a screen has multiple partitions, it should return a FillResponse with just the datasets for the partition that originated the request (i.e., the partition that has the android.app.assist.AssistStructure.ViewNode whose android.app.assist.AssistStructure.ViewNode#isFocused() returns true); then if the user selects a field from a different partition, the Android System will make another #onFillRequest(FillRequest, CancellationSignal, FillCallback) call for that partition, and so on.

Notice that when the user autofill a partition with the data provided by the service and the user did not change these fields, the autofilled value is sent back to the service in the subsequent calls (and can be obtained by calling android.app.assist.AssistStructure.ViewNode#getAutofillValue()). This is useful in the cases where the service must create datasets for a partition based on the choice made in a previous partition. For example, the 1st response for a screen that have credentials and address partitions could be:

new FillResponse.Builder()
                .addDataset(new Dataset.Builder() // partition 1 (credentials)
                    .setValue(id1, AutofillValue.forText("homer"), createPresentation("homer"))
                    .setValue(id2, AutofillValue.forText("D'OH!"), createPresentation("password for homer"))
                    .build())
                .addDataset(new Dataset.Builder() // partition 1 (credentials)
                    .setValue(id1, AutofillValue.forText("flanders"), createPresentation("flanders"))
                    .setValue(id2, AutofillValue.forText("OkelyDokelyDo"), createPresentation("password for flanders"))
                    .build())
                .setSaveInfo(new SaveInfo.Builder(SaveInfo.SAVE_DATA_TYPE_PASSWORD,
                    new AutofillId[] { id1, id2 })
                        .build())
                .build();

Then if the user selected flanders, the service would get a new #onFillRequest(FillRequest, CancellationSignal, FillCallback) call, with the values of the fields id1 and id2 prepopulated, so the service could then fetch the address for the Flanders account and return the following FillResponse for the address partition:

new FillResponse.Builder()
                .addDataset(new Dataset.Builder() // partition 2 (address)
                    .setValue(id3, AutofillValue.forText("744 Evergreen Terrace"), createPresentation("744 Evergreen Terrace")) // street
                    .setValue(id4, AutofillValue.forText("Springfield"), createPresentation("Springfield")) // city
                    .build())
                .setSaveInfo(new SaveInfo.Builder(SaveInfo.SAVE_DATA_TYPE_PASSWORD | SaveInfo.SAVE_DATA_TYPE_ADDRESS,
                    new AutofillId[] { id1, id2 }) // username and password
                         .setOptionalIds(new AutofillId[] { id3, id4 }) // state and zipcode
                        .build())
                .build();

When the service returns multiple FillResponse, the last one overrides the previous; that's why the SaveInfo in the 2nd request above has the info for both partitions.

"PackageVerification"><h3>Package verification</h3>

When autofilling app-specific data (like username and password), the service must verify the authenticity of the request by obtaining all signing certificates of the app being autofilled, and only fulfilling the request when they match the values that were obtained when the data was first saved &mdash; such verification is necessary to avoid phishing attempts by apps that were sideloaded in the device with the same package name of another app. Here's an example on how to achieve that by hashing the signing certificates:

private String getCertificatesHash(String packageName) throws Exception {
              PackageManager pm = mContext.getPackageManager();
              PackageInfo info = pm.getPackageInfo(packageName, PackageManager.GET_SIGNATURES);
              ArrayList<String> hashes = new ArrayList<>(info.signatures.length);
              for (Signature sig : info.signatures) {
                byte[] cert = sig.toByteArray();
                MessageDigest md = MessageDigest.getInstance("SHA-256");
                md.update(cert);
                hashes.add(toHexString(md.digest()));
              }
              Collections.sort(hashes);
              StringBuilder hash = new StringBuilder();
              for (int i = 0; i < hashes.size(); i++) {
                hash.append(hashes.get(i));
              }
              return hash.toString();
            }

If the service did not store the signing certificates data the first time the data was saved &mdash; for example, because the data was created by a previous version of the app that did not use the Autofill Framework &mdash; the service should warn the user that the authenticity of the app cannot be confirmed (see an example on how to show such warning in the Web security section below), and if the user agrees, then the service could save the data from the signing ceriticates for future use.

"IgnoringViews"><h3>Ignoring views</h3>

If the service find views that cannot be autofilled (for example, a text field representing the response to a Captcha challenge), it should mark those views as ignored by calling FillResponse.Builder#setIgnoredIds(AutofillId...) so the system does not trigger a new #onFillRequest(FillRequest, CancellationSignal, FillCallback) when these views are focused.

"WebSecurity"><h3>Web security</h3>

When handling autofill requests that represent web pages (typically view structures whose root's android.app.assist.AssistStructure.ViewNode#getClassName() is a android.webkit.WebView), the service should take the following steps to verify if the structure can be autofilled with the data associated with the app requesting it:

<ol> <li>Use the android.app.assist.AssistStructure.ViewNode#getWebDomain() to get the source of the document. <li>Get the canonical domain using the Public Suffix List (see example below). <li>Use Digital Asset Links to obtain the package name and certificate fingerprint of the package corresponding to the canonical domain. <li>Make sure the certificate fingerprint matches the value returned by Package Manager (see "Package verification" section above). </ol>

Here's an example on how to get the canonical domain using Guava:

private static String getCanonicalDomain(String domain) {
              InternetDomainName idn = InternetDomainName.from(domain);
              while (idn != null && !idn.isTopPrivateDomain()) {
                idn = idn.parent();
              }
              return idn == null ? null : idn.toString();
            }

"WebSecurityDisclaimer">

If the association between the web domain and app package cannot be verified through the steps above, but the service thinks that it is appropriate to fill persisted credentials that are stored for the web domain, the service should warn the user about the potential data leakage first, and ask for the user to confirm. For example, the service could:

<ol> <li>Create a dataset that requires Dataset.Builder#setAuthentication(android.content.IntentSender) authentication to unlock. <li>Include the web domain in the custom presentation for the Dataset.Builder#setValue(AutofillId, AutofillValue, android.widget.RemoteViews) dataset value. <li>When the user selects that dataset, show a disclaimer dialog explaining that the app is requesting credentials for a web domain, but the service could not verify if the app owns that domain. If the user agrees, then the service can unlock the dataset. <li>Similarly, when adding a SaveInfo object for the request, the service should include the above disclaimer in the SaveInfo.Builder#setDescription(CharSequence). </ol>

This same procedure could also be used when the autofillable data is contained inside an IFRAME, in which case the WebView generates a new autofill context when a node inside the IFRAME is focused, with the root node containing the IFRAME's src attribute on android.app.assist.AssistStructure.ViewNode#getWebDomain(). A typical and legitimate use case for this scenario is a financial app that allows the user to login on different bank accounts. For example, a financial app my_financial_app could use a WebView that loads contents from banklogin.my_financial_app.com, which contains an IFRAME node whose src attribute is login.some_bank.com. When fulfilling that request, the service could add an Dataset.Builder#setAuthentication(android.content.IntentSender) authenticated dataset whose presentation displays "Username for some_bank.com" and "Password for some_bank.com". Then when the user taps one of these options, the service shows the disclaimer dialog explaining that selecting that option would release the login.some_bank.com credentials to the my_financial_app; if the user agrees, then the service returns an unlocked dataset with the some_bank.com credentials.

<b>Note:</b> The autofill service could also add well-known browser apps into an allowlist and skip the verifications above, as long as the service can verify the authenticity of the browser app by checking its signing certificate.

"MultipleStepsSave"><h3>Saving when data is split in multiple screens</h3>

Apps often split the user data in multiple screens in the same activity, specially in activities used to create a new user account. For example, the first screen asks for a username, and if the username is available, it moves to a second screen, which asks for a password.

It's tricky to handle save for autofill in these situations, because the autofill service must wait until the user enters both fields before the autofill save UI can be shown. But it can be done by following the steps below:

<ol> <li>In the first #onFillRequest(FillRequest, CancellationSignal, FillCallback) fill request, the service adds a FillResponse.Builder#setClientState(android.os.Bundle) client state bundle in the response, containing the autofill ids of the partial fields present in the screen. <li>In the second #onFillRequest(FillRequest, CancellationSignal, FillCallback) fill request, the service retrieves the FillRequest#getClientState() client state bundle, gets the autofill ids set in the previous request from the client state, and adds these ids and the SaveInfo#FLAG_SAVE_ON_ALL_VIEWS_INVISIBLE to the SaveInfo used in the second response. <li>In the #onSaveRequest(SaveRequest, SaveCallback) save request, the service uses the proper FillContext fill contexts to get the value of each field (there is one fill context per fill request). </ol>

For example, in an app that uses 2 steps for the username and password fields, the workflow would be:

// On first fill request
             AutofillId usernameId = // parse from AssistStructure;
             Bundle clientState = new Bundle();
             clientState.putParcelable("usernameId", usernameId);
             fillCallback.onSuccess(
               new FillResponse.Builder()
                   .setClientState(clientState)
                   .setSaveInfo(new SaveInfo
                        .Builder(SaveInfo.SAVE_DATA_TYPE_USERNAME, new AutofillId[] {usernameId})
                        .build())
                   .build());

             // On second fill request
             Bundle clientState = fillRequest.getClientState();
             AutofillId usernameId = clientState.getParcelable("usernameId");
             AutofillId passwordId = // parse from AssistStructure
             clientState.putParcelable("passwordId", passwordId);
             fillCallback.onSuccess(
               new FillResponse.Builder()
                   .setClientState(clientState)
                   .setSaveInfo(new SaveInfo
                        .Builder(SaveInfo.SAVE_DATA_TYPE_USERNAME | SaveInfo.SAVE_DATA_TYPE_PASSWORD,
                                 new AutofillId[] {usernameId, passwordId})
                        .setFlags(SaveInfo.FLAG_SAVE_ON_ALL_VIEWS_INVISIBLE)
                        .build())
                   .build());

             // On save request
             Bundle clientState = saveRequest.getClientState();
             AutofillId usernameId = clientState.getParcelable("usernameId");
             AutofillId passwordId = clientState.getParcelable("passwordId");
             List<FillContext> fillContexts = saveRequest.getFillContexts();

             FillContext usernameContext = fillContexts.get(0);
             ViewNode usernameNode = findNodeByAutofillId(usernameContext.getStructure(), usernameId);
             AutofillValue username = usernameNode.getAutofillValue().getTextValue().toString();

             FillContext passwordContext = fillContexts.get(1);
             ViewNode passwordNode = findNodeByAutofillId(passwordContext.getStructure(), passwordId);
             AutofillValue password = passwordNode.getAutofillValue().getTextValue().toString();

             save(username, password);

"Privacy"><h3>Privacy</h3>

The #onFillRequest(FillRequest, CancellationSignal, FillCallback) method is called without the user content. The Android system strips some properties of the android.app.assist.AssistStructure.ViewNode view nodes passed to this call, but not all of them. For example, the data provided in the android.view.ViewStructure.HtmlInfo objects set by android.webkit.WebView is never stripped out.

Because this data could contain PII (Personally Identifiable Information, such as username or email address), the service should only use it locally (i.e., in the app's process) for heuristics purposes, but it should not be sent to external servers.

"FieldClassification"><h3>Metrics and field classification</h3>

The service can call #getFillEventHistory() to get metrics representing the user actions, and then use these metrics to improve its heuristics.

Prior to Android android.os.Build.VERSION_CODES#P, the metrics covered just the scenarios where the service knew how to autofill an activity, but Android android.os.Build.VERSION_CODES#P introduced a new mechanism called field classification, which allows the service to dynamically classify the meaning of fields based on the existing user data known by the service.

Typically, field classification can be used to detect fields that can be autofilled with user data that is not associated with a specific app&mdash;such as email and physical address. Once the service identifies that a such field was manually filled by the user, the service could use this signal to improve its heuristics on subsequent requests (for example, by infering which resource ids are associated with known fields).

The field classification workflow involves 4 steps:

<ol> <li>Set the user data through AutofillManager#setUserData(UserData). This data is cached until the system restarts (or the service is disabled), so it doesn't need to be set for all requests. <li>Identify which fields should be analysed by calling FillResponse.Builder#setFieldClassificationIds(AutofillId...). <li>Verify the results through FillEventHistory.Event#getFieldsClassification(). <li>Use the results to dynamically create Dataset or SaveInfo objects in subsequent requests. </ol>

The field classification is an expensive operation and should be used carefully, otherwise it can reach its rate limit and get blocked by the Android System. Ideally, it should be used just in cases where the service could not determine how an activity can be autofilled, but it has a strong suspicious that it could. For example, if an activity has four or more fields and one of them is a list, chances are that these are address fields (like address, city, state, and zip code).

"CompatibilityMode"><h3>Compatibility mode</h3>

Apps that use standard Android widgets support autofill out-of-the-box and need to do very little to improve their user experience (annotating autofillable views and providing autofill hints). However, some apps (typically browsers) do their own rendering and the rendered content may contain semantic structure that needs to be surfaced to the autofill framework. The platform exposes APIs to achieve this, however it could take some time until these apps implement autofill support.

To enable autofill for such apps the platform provides a compatibility mode in which the platform would fall back to the accessibility APIs to generate the state reported to autofill services and fill data. This mode needs to be explicitly requested for a given package up to a specified max version code allowing clean migration path when the target app begins to support autofill natively. Note that enabling compatibility may degrade performance for the target package and should be used with caution. The platform supports creating an allowlist for including which packages can be targeted in compatibility mode to ensure this mode is used only when needed and as long as needed.

You can request compatibility mode for packages of interest in the meta-data resource associated with your service. Below is a sample service declaration:

<service android:name=".MyAutofillService"
                         android:permission="android.permission.BIND_AUTOFILL_SERVICE">
                <intent-filter>
                    <action android:name="android.service.autofill.AutofillService" />
                </intent-filter>
                <meta-data android:name="android.autofill" android:resource="@xml/autofillservice" />
            </service>

In the XML file you can specify one or more packages for which to enable compatibility mode. Below is a sample meta-data declaration:

<autofill-service xmlns:android="http://schemas.android.com/apk/res/android">
                <compatibility-package android:name="foo.bar.baz" android:maxLongVersionCode="1000000000"/>
            </autofill-service>

Notice that compatibility mode has limitations such as: <ul> <li>No manual autofill requests. Hence, the FillRequestFillRequest#getFlags() flags never have the FillRequest#FLAG_MANUAL_REQUEST flag. <li>The value of password fields are most likely masked&mdash;for example, **** instead of 1234. Hence, you must be careful when using these values to avoid updating the user data with invalid input. For example, when you parse the FillRequest and detect a password field, you could check if its android.app.assist.AssistStructure.ViewNode#getInputType() input type has password flags and if so, don't add it to the SaveInfo object. <li>The autofill context is not always AutofillManager#commit() committed when an HTML form is submitted. Hence, you must use other mechanisms to trigger save, such as setting the SaveInfo#FLAG_SAVE_ON_ALL_VIEWS_INVISIBLE flag on SaveInfo.Builder#setFlags(int) or using SaveInfo.Builder#setTriggerId(AutofillId). <li>Browsers often provide their own autofill management system. When both the browser and the platform render an autofill dialog at the same time, the result can be confusing to the user. Such browsers typically offer an option for users to disable autofill, so your service should also allow users to disable compatiblity mode for specific apps. That way, it is up to the user to decide which autofill mechanism&mdash;the browser's or the platform's&mdash;should be used. </ul>

Java documentation for android.service.autofill.AutofillService.

Portions of this page are modifications based on work created and shared by the Android Open Source Project and used according to terms described in the Creative Commons 2.5 Attribution License.

Constructors

Fields

Properties

Methods

Explicit Interface Implementations

Extension Methods

Applies to