1# external_updater
2
3external updater is a tool to automatically update libraries in external/.
4
5The documentation on this page is for users of `external_updater`. If you're
6looking for developer docs, see [docs/dev.md](docs/dev.md).
7
8## Usage
9
10In each of the examples below, `$PROJECT_PATH` is the path to the project to
11operate on. If more than one path is given, external_updater will operate on
12each in turn.
13
14Note: Older versions of external_updater used a different path resolution
15method. Relative paths were resolved relative to `//external` rather than the
16CWD, which meant tab-completed paths would only work if the CWD was
17`//external`, and that wildcards had to be escaped for processing by
18external_updater rather than the shell (e.g.
19`updater.sh 'check rust/crates/*'`). That behavior was removed to support CWD
20relative paths. If you want the old behavior back, leave a comment on
21http://b/243685332 or https://r.android.com/2855445.
22
23Check updates for a library or verify METADATA is valid:
24
25```shell
26tools/external_updater/updater.sh check $PROJECT_PATH
27```
28
29Update a library, commit, and upload the change to Gerrit:
30
31```shell
32tools/external_updater/updater.sh update $PROJECT_PATH
33```
34
35Update a library without committing and uploading to Gerrit:
36
37```shell
38tools/external_updater/updater.sh update --no-upload $PROJECT_PATH
39```
40
41Update a library on top of the local changes in the current branch, commit, and upload the change to Gerrit:
42
43```shell
44tools/external_updater/updater.sh update --keep-local-changes $PROJECT_PATH
45```
46
47Update a library without building:
48
49```shell
50tools/external_updater/updater.sh update --no-build $PROJECT_PATH
51```
52
53Update a library and add bug number to the commit message:
54
55```shell
56tools/external_updater/updater.sh update --bug $BUG_NUMBER $PROJECT_PATH
57```
58
59PROJECT_PATH can be the path to a library under external/, e.g.
60external/kotlinc, or external/python/cpython3.
61
62## Configure
63
64To use this tool, a METADATA file must present at the root of the
65repository. The full definition can be found
66[here](https://android.googlesource.com/platform/tools/external_updater/+/refs/heads/main/metadata.proto).
67Or see example [here](https://android.googlesource.com/platform/external/ImageMagick/+/refs/heads/main/METADATA)
68
69The most important part in the file is a list of urls.
70`external_updater` will go through all urls and uses the first
71supported url.
72
73### Git upstream
74
75If type of a URL is set to GIT, the URL must be a git upstream
76(the one you can use with `git clone`). And the version field must
77be either a version tag, or SHA. The tool will find the latest
78version tag or sha based on it.
79
80When upgrade, the tool will simply run `git merge tag/sha`.
81
82IMPORTANT: It is suggested to set up a `upstream-main` branch to
83replicate upstream. Because most users don't have the privilege to
84upload changes not authored by themselves. This can be done by
85filing a bug to componentid:99104.
86
87#### SHA
88
89If the version is a SHA, the tool will always try to upgrade to the
90top of upstream. As long as there is any new change upstream, local
91library will be treated as stale.
92
93#### Version tag
94
95If the version is not a SHA, the tool will try to parse the version
96to get a numbered version. Currently the supported version format is:
97
98```markdown
99<prefix><version_number><suffix>
100```
101
102version_number part can be numbers separated by `.` or `-` or `_`.
103
104If you have project where this isn't working, file a bug so we can take a look.
105
106#### Local changes
107
108It is suggested to verify all local changes when upgrading. This can
109be done easily in Gerrit, by comparing parent2 and the patchset.
110
111
112### GitHub archive
113
114If the url type is ARCHIVE, and the url is from GitHub, `external_updater`
115can upgrade a library based on GitHub releases.
116
117If you have the choice between archives and git tags, choose tags.
118Because that makes it easier to manage local changes.
119
120The tool will query GitHub to get the latest release from:
121
122```url
123https://github.com/user/proj/releases/latest
124```
125
126If the tag of latest release is not equal to version in METADATA file, a
127new version is found. The tool will download the tarball and overwrite the
128library with it.
129
130If there are multiple archives in one GitHub release, the one most
131[similar](https://en.wikipedia.org/wiki/Edit_distance) to previous
132(from METADATA) will be used.
133
134After upgrade, files not present in the new tarball will be removed. But we
135explicitly keep files famous in Android tree.
136See [here](https://android.googlesource.com/platform/tools/external_updater/+/refs/heads/main/update_package.sh).
137
138If more files need to be reserved, a post_update.sh can be created to copy
139these files over.
140See [example](https://android.googlesource.com/platform/external/kotlinc/+/refs/heads/main/post_update.sh).
141
142#### Local patches
143
144Local patches can be kept as patches/*.diff. They will be applied after
145upgrade. [example](https://cs.android.com/android/platform/superproject/+/main:external/jsmn/patches/header.diff)
146
147## Email notification
148
149There is some support to automatically check updates for all external
150libraries every hour, send email and change. Currently this is done by
151running the following script on a desktop machine.
152
153```shell
154#!/bin/bash
155
156cd /src/aosp
157while true
158do
159        repo abandon tmp_auto_upgrade
160        repo forall -c git checkout .
161        repo forall -c git clean -xdf
162        repo sync -c
163        source build/envsetup.sh
164        lunch aosp_arm-eng
165        mmma tools/external_updater
166
167        out/soong/host/linux-x86/bin/external_updater_notifier \
168                --history ~/updater/history \
169                --recipients=android_external_lib_updates@google.com \
170                --generate_change \
171                --all
172        date
173        echo "Sleeping..."
174        sleep 3600
175done
176```
177