簡介
AccessibilityService的設計初衷是為了輔助有身體缺陷的群體使用Android應用，它的設計貫穿著Android的控件樹View, ViewGroup, ViewRootImpl體系。借助于system_server進程的中轉，能夠注冊Accessibility事件的客戶端可以具備通過system_server提供的Accessibility服務來實現監聽、操作其它應用視圖的功能。這個功能十分強大，可以模擬用戶的行為去操作其它APP，常常被用在自動化測試、微信搶紅包、自動回復等功能實現中。
寫這個的初衷有二：
- 之前已經完成了Android View控件樹的繪制、事件分發的源碼分析，知識儲備足夠
- 最近接觸到了一些自動化方面的項目，并且對使用無障礙服務實現的自動微信搶紅包功能原理十分好奇
整體圖
類圖
- AccessibilityService: APP端直接繼承的類，本質上是Service，通過onBind獲取匿名Binder對象實現通信
- IAccessibilityServiceClientWrapper: 用于和system_server通信的匿名Binder服務
- AccessibilityInteractionClient: 本質上是個binder服務，用于獲取Node信息
- AccessibilityManagerService: 運行在system_server的實名binder服務，是整體的管理類
- Service: AccessibilityManagerService的內部類，用于響應AccessibilityInteractionClient的binder通信請求
- AccessibilityInteractionConnection: 運行在被監測的APP端，提供查找、點擊視圖等服務
- AccessibilityManager: 運行在各個APP端，用于發送視圖變化事件
- AccessibilityInteractionController: 具體視圖查找、點擊服務的中間控制器
- AccessibilityNodeProvider: 由客戶端實現的視圖節點內容提供者，最終操作的實現者
整體設計圖
實例代碼
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
|
public class AutoDismissService extends AccessibilityService { @Override public void onAccessibilityEvent(AccessibilityEvent event) { if (event == null) { return; } // 自動將android系統彈出的其它crash dialog取消 dismissAppErrorDialogIfExists(event); } private void dismissAppErrorDialogIfExists(AccessibilityEvent event) { // WINDOW視圖變化才進行對應操作 if ((event.getEventType() == AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED && event.getPackageName().equals("android")) { // 查找帶有"OK"字符的可點擊Node AccessibilityNodeInfo nodeInfo = findViewByText("OK", true); if (nodeInfo != null) { // 查找到后執行點擊操作 performViewClick(nodeInfo); } } public AccessibilityNodeInfo findViewByText(String text, boolean clickable) { // 獲取當前窗口父節點 AccessibilityNodeInfo accessibilityNodeInfo = getRootInActiveWindow(); if (accessibilityNodeInfo == null) { return null; } // 獲取到滿足字符要求的節點 List<AccessibilityNodeInfo> nodeInfoList = accessibilityNodeInfo.findAccessibilityNodeInfosByText(text); if (nodeInfoList != null && !nodeInfoList.isEmpty()) { for (AccessibilityNodeInfo nodeInfo : nodeInfoList) { if (nodeInfo != null && (nodeInfo.isClickable() == clickable)) { return nodeInfo; } } } return null; } public void performViewClick(AccessibilityNodeInfo nodeInfo) { if (nodeInfo == null) { return; } // 由下至上進行查詢，直到尋找到可點擊的節點 while (nodeInfo != null) { if (nodeInfo.isClickable()) { nodeInfo.performAction(AccessibilityNodeInfo.ACTION_CLICK); break; } nodeInfo = nodeInfo.getParent(); } }} |
以上是一個典型的實現Accessibility功能的JAVA代碼，主要涉及三點功能：
- 當系統中有應用視圖變化后，onAccessibilityEvent 方法會自動被system_server調用
- 通過AccessibilityService的getRootInActiveWindow與findAccessibilityNodeInfosByText方法，可以獲取到節點信息
- 通過AccessibilityNodeInfo的performAction方法，最終會在被監聽APP中執行對應操作
本篇文章將會圍繞著這三點主要功能進行源碼分析
源碼分析
常見 AccessibilityEvent 事件種類
| 序號 | 種類名稱 | 觸發時機 |
|---|---|---|
| 1 | TYPE_VIEW_CLICKED | 可點擊的組件被點擊 |
| 2 | TYPE_VIEW_LONG_CLICKED | 可點擊的組件被長按 |
| 3 | TYPE_VIEW_SELECTED | 組件被選中 |
| 4 | TYPE_VIEW_FOCUSED | 組件獲取到了焦點 |
| 5 | TYPE_VIEW_TEXT_CHANGED | 組件中的文本發生變化 |
| 6 | TYPE_VIEW_SCROLLED | 組件被滑動 |
| 7 | TYPE_WINDOW_STATE_CHANGED | dialog等被打開 |
| 8 | TYPE_NOTIFICATION_STATE_CHANGED | 通知彈出 |
| 9 | TYPE_WINDOW_CONTENT_CHANGED | 組件樹發生了變化 |
onAccessibilityEvent 觸發流程
這里以TextView.setText觸發事件變化流程為例進行分析
TextView.setText
應用組件狀態發生變化
frameworks/base/core/java/android/widget/TextView.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
|
private void setText(CharSequence text, BufferType type, boolean notifyBefore, int oldlen) { ... notifyViewAccessibilityStateChangedIfNeeded(AccessibilityEvent.CONTENT_CHANGE_TYPE_TEXT); ... }public void notifyViewAccessibilityStateChangedIfNeeded(int changeType) { if (!AccessibilityManager.getInstance(mContext).isEnabled() || mAttachInfo == null) { return; } if (mSendViewStateChangedAccessibilityEvent == null) { // 本質上是一個Runnable，意味著這里的流程會進入異步處理 mSendViewStateChangedAccessibilityEvent = new SendViewStateChangedAccessibilityEvent(); } mSendViewStateChangedAccessibilityEvent.runOrPost(changeType);}private class SendViewStateChangedAccessibilityEvent implements Runnable { ... @Override public void run() { mPosted = false; mPostedWithDelay = false; mLastEventTimeMillis = SystemClock.uptimeMillis(); if (AccessibilityManager.getInstance(mContext).isEnabled()) { final AccessibilityEvent event = AccessibilityEvent.obtain(); event.setEventType(AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED); event.setContentChangeTypes(mChangeTypes); // 最終TYPE_WINDOW_CONTENT_CHANGED事件在這里異步發送 sendAccessibilityEventUnchecked(event); } mChangeTypes = 0; } ...}public void sendAccessibilityEventUnchecked(AccessibilityEvent event) { if (mAccessibilityDelegate != null) { mAccessibilityDelegate.sendAccessibilityEventUnchecked(this, event); } else { sendAccessibilityEventUncheckedInternal(event); }}public void sendAccessibilityEventUnchecked(View host, AccessibilityEvent event) { host.sendAccessibilityEventUncheckedInternal(event);}public void sendAccessibilityEventUncheckedInternal(AccessibilityEvent event) { if (!isShown()) { return; } ... // 此處交由TextView所在父View進行處理，為責任鏈模式，事件經過層層向上傳遞，最終交由ViewRootImpl進行處理 ViewParent parent = getParent(); if (parent != null) { getParent().requestSendAccessibilityEvent(this, event); }} |
ViewRootImpl.requestSendAccessibilityEvent
ViewRootImpl將事件派發到system_server
frameworks/base/core/java/android/view/ViewRootImpl.java
|
1
2
3
4
5
6
7
|
@Overridepublic boolean requestSendAccessibilityEvent(View child, AccessibilityEvent event) { ... // 本地調用到AccessibilityManager進行事件發送 mAccessibilityManager.sendAccessibilityEvent(event); return true;} |
frameworks/base/core/java/android/view/accessibility/AccessibilityManager.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
public void sendAccessibilityEvent(AccessibilityEvent event) { final IAccessibilityManager service; final int userId; synchronized (mLock) { // 獲取system_server的Accessibility實名服務 service = getServiceLocked(); ... } try { ... long identityToken = Binder.clearCallingIdentity(); // binder call 到服務端，進行事件分發中轉 doRecycle = service.sendAccessibilityEvent(event, userId); Binder.restoreCallingIdentity(identityToken); ... } catch (RemoteException re) { Log.e(LOG_TAG, "Error during sending " + event + " ", re); } finally { ... }} |
AccessibilityManagerService.sendAccessibilityEvent
system_server將事件分發到各個監聽組件變化的Service
frameworks/base/services/accessibility/java/com/android/server/accessibility/AccessibilityManagerService.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
|
// binder call 到服務端，觸發事件派發@Overridepublic boolean sendAccessibilityEvent(AccessibilityEvent event, int userId) { synchronized (mLock) { ... if (mSecurityPolicy.canDispatchAccessibilityEventLocked(event)) { ... notifyAccessibilityServicesDelayedLocked(event, false); notifyAccessibilityServicesDelayedLocked(event, true); } ... } return (OWN_PROCESS_ID != Binder.getCallingPid());}private void notifyAccessibilityServicesDelayedLocked(AccessibilityEvent event, boolean isDefault) { try { UserState state = getCurrentUserStateLocked(); for (int i = 0, count = state.mBoundServices.size(); i < count; i++) { Service service = state.mBoundServices.get(i); if (service.mIsDefault == isDefault) { if (canDispatchEventToServiceLocked(service, event)) { // 調用內部服務，以觸發事件派發 service.notifyAccessibilityEvent(event); } } } } catch (IndexOutOfBoundsException oobe) { ... }}class Service extends IAccessibilityServiceConnection.Stub implements ServiceConnection, DeathRecipient { public void notifyAccessibilityEvent(AccessibilityEvent event) { synchronized (mLock) { ... if ((mNotificationTimeout > 0) && (eventType != AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED)) { ... // 按照慣例，異步分發到客戶端進行派發 message = mEventDispatchHandler.obtainMessage(eventType); } else { message = mEventDispatchHandler.obtainMessage(eventType, newEvent); } mEventDispatchHandler.sendMessageDelayed(message, mNotificationTimeout); } } }public Handler mEventDispatchHandler = new Handler(mMainHandler.getLooper()) { @Override public void handleMessage(Message message) { final int eventType = message.what; AccessibilityEvent event = (AccessibilityEvent) message.obj; notifyAccessibilityEventInternal(eventType, event); }};private void notifyAccessibilityEventInternal(int eventType, AccessibilityEvent event) { IAccessibilityServiceClient listener; ... // mServiceInterface是通過bind客戶端的AccessibilityService，在onServiceConnected連接成功后，獲取到binder proxy轉化來的，以這種方式實現了system_server與客戶端的通信 listener = mServiceInterface; ... try { listener.onAccessibilityEvent(event); if (DEBUG) { Slog.i(LOG_TAG, "Event " + event + " sent to " + listener); } } catch (RemoteException re) { Slog.e(LOG_TAG, "Error during sending " + event + " to " + listener, re); } finally { event.recycle(); }} |
AccessibilityService.onAccessibilityEvent
APP接收到組件變化的事件，并可以選擇做出相應的處理
frameworks/base/core/java/android/accessibilityservice/AccessibilityService.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
|
// 抽象方法，模板模式，被系統主動調用public abstract void onAccessibilityEvent(AccessibilityEvent event);// 該service是被system_server主動綁定的，獲取到IAccessibilityServiceClientWrapper的proxy來實現系統的主動調用@Overridepublic final IBinder onBind(Intent intent) { return new IAccessibilityServiceClientWrapper(this, getMainLooper(), new Callbacks() { ... @Override public void onAccessibilityEvent(AccessibilityEvent event) { AccessibilityService.this.onAccessibilityEvent(event); } ... }}// 收到binder調用后，使用handler異步進行事件的處理public void onAccessibilityEvent(AccessibilityEvent event) { Message message = mCaller.obtainMessageO(DO_ON_ACCESSIBILITY_EVENT, event); mCaller.sendMessage(message);}@Overridepublic void executeMessage(Message message) { switch (message.what) { case DO_ON_ACCESSIBILITY_EVENT: { AccessibilityEvent event = (AccessibilityEvent) message.obj; if (event != null) { AccessibilityInteractionClient.getInstance().onAccessibilityEvent(event); // 通過回調調用以觸發事件 mCallback.onAccessibilityEvent(event); ... } } return; }} |
getRootInActiveWindow 父節點獲取流程
在調用findAccessibilityNodeInfosByText之前，需要通過getRootInActiveWindow方法獲取到父節點，才能通過調用父AccessibilityNodeInfo的方法進行其子節點信息查詢
AccessibilityService.getRootInActiveWindow
frameworks/base/core/java/android/accessibilityservice/AccessibilityService.java
|
1
2
3
4
|
public AccessibilityNodeInfo getRootInActiveWindow() { // 查找父節點的操作沒有在自己的類中實現，而是交由了同一進程的Client管理類進行處理 return AccessibilityInteractionClient.getInstance().getRootInActiveWindow(mConnectionId);} |
frameworks/base/core/java/android/view/accessibility/AccessibilityInteractionClient.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
public AccessibilityNodeInfo getRootInActiveWindow(int connectionId) { return findAccessibilityNodeInfoByAccessibilityId(connectionId, AccessibilityNodeInfo.ACTIVE_WINDOW_ID, AccessibilityNodeInfo.ROOT_NODE_ID, false, AccessibilityNodeInfo.FLAG_PREFETCH_DESCENDANTS);}public AccessibilityNodeInfo findAccessibilityNodeInfoByAccessibilityId(int connectionId, int accessibilityWindowId, long accessibilityNodeId, boolean bypassCache, int prefetchFlags) { ... // 嘗試binder call到system_server，請求中轉到其它APP進程中查詢父節點信息，注意的是這里AccessibilityInteractionClient本身是個binder服務端，把this傳到system_server后，其它進程可以通過這個引用拿到binder proxy，以實現通信 final boolean success = connection.findAccessibilityNodeInfoByAccessibilityId( accessibilityWindowId, accessibilityNodeId, interactionId, this, prefetchFlags, Thread.currentThread().getId()); Binder.restoreCallingIdentity(identityToken); // If the scale is zero the call has failed. if (success) { // 調用成功后，這里會嘗試同步獲取結果 List<AccessibilityNodeInfo> infos = getFindAccessibilityNodeInfosResultAndClear( interactionId); finalizeAndCacheAccessibilityNodeInfos(infos, connectionId); if (infos != null && !infos.isEmpty()) { return infos.get(0); } } ... } |
Service.findAccessibilityNodeInfoByAccessibilityId
注意一下，這里的Service不是Android中的四大組件的Service，取名叫AccessiblitManagerServiceInternal其實更合適
frameworks/base/services/accessibility/java/com/android/server/accessibility/AccessibilityManagerService.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
@Overridepublic boolean findAccessibilityNodeInfoByAccessibilityId( int accessibilityWindowId, long accessibilityNodeId, int interactionId, IAccessibilityInteractionConnectionCallback callback, int flags, long interrogatingTid) throws RemoteException { ... // 獲取到其他APP的節點獲取服務 IAccessibilityInteractionConnection connection = null; ... resolvedWindowId = resolveAccessibilityWindowIdLocked(accessibilityWindowId); ... if (!permissionGranted) { return false; } else { connection = getConnectionLocked(resolvedWindowId); if (connection == null) { return false; } } ... // 這里的callback為之前應用的服務proxy句柄，將它傳入是為了之后的信息通信不再需要經過system_server中轉，而是直接可以APP對APP的進行通信 connection.findAccessibilityNodeInfoByAccessibilityId(accessibilityNodeId, partialInteractiveRegion, interactionId, callback, mFetchFlags | flags, interrogatingPid, interrogatingTid, spec); ...} |
AccessibilityInteractionConnection.findAccessibilityNodeInfoByAccessibilityId
這里調用到了APP端，其實同onAccessibilityEvent調用流程一樣，是APP->SYSTEM->APP的調用順序
frameworks/base/core/java/android/view/ViewRootImpl.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
@Overridepublic void findAccessibilityNodeInfoByAccessibilityId(long accessibilityNodeId, Region interactiveRegion, int interactionId, IAccessibilityInteractionConnectionCallback callback, int flags, int interrogatingPid, long interrogatingTid, MagnificationSpec spec) { ViewRootImpl viewRootImpl = mViewRootImpl.get(); if (viewRootImpl != null && viewRootImpl.mView != null) { // 這里也只是委托給控制類進行細節操作的處理 viewRootImpl.getAccessibilityInteractionController() .findAccessibilityNodeInfoByAccessibilityIdClientThread(accessibilityNodeId, interactiveRegion, interactionId, callback, flags, interrogatingPid, interrogatingTid, spec); } else { ... }} |
frameworks/base/core/java/android/view/AccessibilityInteractionController.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
|
private void findAccessibilityNodeInfoByAccessibilityIdUiThread(Message message) { ... // 初始化將會返回的節點 List<AccessibilityNodeInfo> infos = mTempAccessibilityNodeInfoList; infos.clear(); try { if (mViewRootImpl.mView == null || mViewRootImpl.mAttachInfo == null) { return; } mViewRootImpl.mAttachInfo.mAccessibilityFetchFlags = flags; View root = null; if (accessibilityViewId == AccessibilityNodeInfo.UNDEFINED_ITEM_ID) { root = mViewRootImpl.mView; } else { root = findViewByAccessibilityId(accessibilityViewId); } ... } finally { try { ... adjustIsVisibleToUserIfNeeded(infos, interactiveRegion); // 通過callback binder proxy句柄，將節點信息binder回應用 callback.setFindAccessibilityNodeInfosResult(infos, interactionId); infos.clear(); } catch (RemoteException re) { /* ignore - the other side will time out */ } ... }} |
AccessibilityInteractionClient.setFindAccessibilityNodeInfosResult
frameworks/base/core/java/android/view/accessibility/AccessibilityInteractionClient.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
public void setFindAccessibilityNodeInfosResult(List<AccessibilityNodeInfo> infos, int interactionId) { synchronized (mInstanceLock) { if (interactionId > mInteractionId) { if (infos != null) { ... // 設置應用的返回節點信息 if (!isIpcCall) { mFindAccessibilityNodeInfosResult = new ArrayList<>(infos); } else { mFindAccessibilityNodeInfosResult = infos; } } else { mFindAccessibilityNodeInfosResult = Collections.emptyList(); } mInteractionId = interactionId; } // 釋放鎖，停止等待，節點信息已經取回 mInstanceLock.notifyAll(); }} |
findAccessibilityNodeInfosByText與performAction 對目標節點進行操作
AccessibilityNodeInfo.findAccessibilityNodeInfosByText
找到父節點信息后，就可以通過父節點獲取對應的子節點信息了
frameworks/base/core/java/android/view/accessibility/AccessibilityNodeInfo.java
|
1
2
3
4
5
6
7
8
|
public List<AccessibilityNodeInfo> findAccessibilityNodeInfosByText(String text) { ... // 一樣的流程，通過AccessibilityInteractionClient去獲取信息 AccessibilityInteractionClient client = AccessibilityInteractionClient.getInstance(); return client.findAccessibilityNodeInfosByText(mConnectionId, mWindowId, mSourceNodeId, text);}``` |
以下的代碼流程同getRootInActiveWindow大概一致，就不詳細分析了
#### AccessibilityNodeInfo.performAction
獲取到對應子節點后，通過performAction可以執行對應的操作了，如常用的點擊
最終回調用到AccessibilityInteractionController,獲取到AccessibilityProvier后就可以執行performAction的最終操作了
frameworks/base/core/java/android/view/AccessibilityInteractionController.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
```javaprivate void performAccessibilityActionUiThread(Message message) { View target = null; if (accessibilityViewId != AccessibilityNodeInfo.UNDEFINED_ITEM_ID) { target = findViewByAccessibilityId(accessibilityViewId); } else { target = mViewRootImpl.mView; } if (target != null && isShown(target)) { AccessibilityNodeProvider provider = target.getAccessibilityNodeProvider(); if (provider != null) { if (virtualDescendantId != AccessibilityNodeInfo.UNDEFINED_ITEM_ID) { // 在客戶端執行performAction操作 succeeded = provider.performAction(virtualDescendantId, action, arguments); } else { succeeded = provider.performAction(AccessibilityNodeProvider.HOST_VIEW_ID, action, arguments); } } else if (virtualDescendantId == AccessibilityNodeInfo.UNDEFINED_ITEM_ID) { succeeded = target.performAccessibilityAction(action, arguments); } }} |
frameworks/base/core/java/android/view/View.java
|
1
2
3
4
5
6
7
8
9
10
11
12
|
public boolean performAccessibilityActionInternal(int action, Bundle arguments) { ... switch (action) { case AccessibilityNodeInfo.ACTION_CLICK: { if (isClickable()) { // 最終調用到我們熟悉的View.performClick方法 performClick(); return true; } } break; ...} |
分析到這里可以看到，Accessibility服務框架類似于hook在Android View組件樹中的一套實現，它并不是獨立的一套機制，而是”寄生”在View的顯示、事件分發的流程中。
總結
功能實現依賴于ViewRootImpl, ViewGroup, View視圖層級管理的基本架構。在視圖變化時發出事件、當收到視圖操作請求時也能夠作出響應。
system_server在實現該功能的過程中扮演著中間人的角色。當被監聽APP視圖變化時，APP首先會發出事件到system_server，隨后再中轉到監聽者APP端。當監聽者APP想要執行視圖操作時，也是首先在system_server中找到對應的客戶端binder proxy，再調用相應接口調用到被監聽APP中。完成相關操作后，通過已經獲取到的監聽APP binder proxy句柄，直接binder call到對應的監聽客戶端。
無障礙權限十分重要，切記不可濫用，APP自身也需要有足夠的安全意識，防止惡意應用通過該服務獲取用戶隱私信息
好了，以上就是這篇文章的全部內容了，希望本文的內容對大家的學習或者工作具有一定的參考學習價值，如果有疑問大家可以留言交流，謝謝大家對服務器之家的支持。
原文鏈接：http://navyblue.top/2018/06/10/從源碼角度看AccessibilityService/
