Openharmony光标不显示问题
编辑
2025-01-22
工作知识
0
请注意，本文编写于 108 天前，最后修改于 108 天前，其中某些信息可能已经过时。

目录

一、问题定位
二、代码梳理
2.1 inputdevicemanager
2.2 inputwindowsmanager
2.3 pointerdrawingmanager
三、定位问题
3.1 单独编译模块
3.2 更新和替换库
3.3 服务控制
3.4 hilog查看
3.5 参考社区pr
四、代码改动
五、总结

Openharmony在使用hdmi显示时，发现接上鼠标之后，光标并不显示，于是开始定位，如下是相关分析解释

一、问题定位

Openharmony4.0的rk3588系统上，插上hdmi后显示，鼠标不亮，不方便非触摸屏的设备进行开发使用。

根据openharmony代码熟悉，根据文章《Openharmony目录树》可以知道其可能涉及子系统代码位置在foundation/multimodalinput/位置，如下图

image.png 根据BUILD.gn可以发现mmi_tests工具可以测试光标事件，但是实际上我们存在光标事件，只是光标不显示，故此工具不适用此问题，只能从代码角度分析

根据sa配置可以知道，multimodalinput服务关键服务是libmmi-server.so，如下

# cat /system/profile/multimodalinput.json 
{
 "process": "multimodalinput",
 "systemability": [
 {
 "name": 3101,
 "libpath": "libmmi-server.z.so",
 "run-on-create": true,
 "distributed": true,
 "dump_level": 1
 }
 ]

根据service/BUILD.gn可以发现，与libmmi-server.so相关的代码如下

ohos_shared_library("libmmi-server") {
 sources = libmmi_service_sources
......
 if (input_feature_mouse) {
 sources += [
 "mouse_event_normalize/src/mouse_device_state.cpp",
 "mouse_event_normalize/src/mouse_event_normalize.cpp",
 "mouse_event_normalize/src/mouse_transform_processor.cpp",
 "touch_event_normalize/src/gesture_transform_processor.cpp",
 "touch_event_normalize/src/touchpad_transform_processor.cpp",
 ]
 if (input_feature_pointer_drawing) {
 sources += [ "window_manager/src/pointer_drawing_manager.cpp" ]
 } else {
 sources += [ "window_manager/src/i_pointer_drawing_manager.cpp" ]
 }
 }
libmmi_service_sources = [
 "$mmi_path/frameworks/proxy/event_handler/src/bytrace_adapter.cpp",
 "delegate_task/src/delegate_tasks.cpp",
 "device_config/src/device_config_file_parser.cpp",
 "device_manager/src/input_device_manager.cpp",
 "dfx/src/dfx_hisysevent.cpp",
 "display_state_manager/src/display_event_monitor.cpp",
 "event_dispatch/src/event_dispatch_handler.cpp",
 "event_dump/src/event_dump.cpp",
 "event_handler/src/anr_manager.cpp",
 "event_handler/src/event_normalize_handler.cpp",
 "event_handler/src/input_event_handler.cpp",
 "event_handler/src/key_auto_repeat.cpp",
 "event_handler/src/key_event_value_transformation.cpp",
 "event_handler/src/key_map_manager.cpp",
 "fingersense_wrapper/src/fingersense_wrapper.cpp",
 "libinput_adapter/src/hotplug_detector.cpp",
 "libinput_adapter/src/libinput_adapter.cpp",
 "message_handle/src/server_msg_handler.cpp",
 "module_loader/src/mmi_service.cpp",
 "module_loader/src/uds_server.cpp",
 "permission_helper/src/permission_helper.cpp",
 "timer_manager/src/timer_manager.cpp",
 "window_manager/src/input_display_bind_helper.cpp",
 "window_manager/src/input_windows_manager.cpp",
 ]

于是阅读代码和定位逻辑，可以发现输入设备的识别和管理在input_device_manager.cpp实现

foundation/multimodalinput/input/service/device_manager/src/input_device_manager.cpp

而光标图层管理的代码在input_windows_manager.cpp实现

foundation/multimodalinput/input/service/window_manager/src/input_windows_manager.cpp

实际的光标绘制的代码在pointer_drawing_manager.cpp实现

foundation/multimodalinput/input/service/window_manager/src/pointer_drawing_manager.cpp

二、代码梳理

相关问题定位清晰后，随针对此三块代码进行分析和调试。

2.1 input_device_manager

此文件功能作为输入设备的识别，添加，其关键函数如下，详细代码实现不必列出：

GetInputDevice	实例化InputDevice，返回InputDevice对象
FillInputDevice	填充InputDevice对象的数据
OnInputDeviceAdded	通过libinput库获得设备信息，并分发光标绘制任务
ScanPointerDevice	遍历InputDevice实例，找到pointer设备
IsPointerDevice	如果libinput_device_get_tags存在EVDEV_UDEV_TAG_MOUSE，则是pointer设备

2.2 input_windows_manager

此文件功能作为输入设备的图层显示设置，窗口信息，光标事件。其关键函数如下

GetClientFd	根据pointer事件获取client的fd值，用作后续获取session的句柄
GetDisplayId	根据pointer事件获取displayid
UpdateDisplayInfo	通过displayGroupInfo更新displayid和WindowInfo
SendPointerEvent	通过session发送点击事件
DispatchPointer	通过session发送点击事件（存在上次点击事件的情况）
GetPhysicalDisplayCoord	通过libinput获取触摸的坐标
SetPointerStyle	设置光标style
GetPointerStyle	获取光标style
SelectWindowInfo	根据点击事件获取windowid，从而获得WindowInfo
UpdateMouseTarget	更新鼠标位置和显示
UpdateTargetPointer	根据pointer的类型选择更新对应事件位置和显示
UpdateAndAdjustMouseLocation	调整鼠标位置

2.3 pointer_drawing_manager

此文件处理真正的光标绘制需求，根据光标的位置和style来执行绘制请求

DrawPointer	绘制光标
InitLayer	初始化光标层
CreatePointerWindow	创建surface节点用于绘制光标层
GetSurfaceBuffer	根据传入的surface节点，获取sufface buffer
DrawPixelmap	根据icon使用canvas绘制bitmap
SetMouseIcon	设置鼠标icon
DecodeImageToPixelMap	将icon解码成pixelmap
SetPointerColor	设置光标颜色
SetPointerSize	设置光标大小
UpdatePointerDevice	更新光标设备
DrawManager	绘制管理，根据是否是光标设备来判断是否绘制和销毁窗口
UpdatePointerVisible	更新光标是否可见
SetPointerStyle	设置光标style
DrawPointerStyle	绘制光标style

三、定位问题

在完成上述1.判断问题相关代码，2.熟悉代码流程之后，可以开始进行定位实际问题，主要策略如下

3.1 单独编译模块

根据解析BUILD.gn可以知道，此模块可以单独编译，如下是单独编译命令

prebuilts/build-tools/linux-x86/bin/ninja -w dupbuild=warn -C out/rk3588/ libmmi-server

3.2 更新和替换库

可以将编译成功的库文件，替换到系统内，通过hdc命令，如下

hdc -t 9b01005932503033320045da20422900 shell mount -o remount,rw /
hdc -t 9b01005932503033320045da20422900 file send /root/tf/rk3588/out/rk3588/multimodalinput/input/libmmi-server.z.so /system/lib64/

3.3 服务控制

根据上述措施，已经成功将libmmi-server推送到系统内部，这时候需要将multimodalinput服务关闭和拉起，方法如下

service_control stop multimodalinput
service_control start multimodalinput

当然，因为openharmony的服务可以自动拉起，所以直接kill也可以，如下

kill -9 `pidof multimodalinput`

3.4 hilog查看

对于调试过程中，主要是通过hilog来进行分析，本例子用到如下

1、设置hilog为调试

hilog -b D

2、定位InputDeviceManager日志

hilog -T InputDeviceManager

3、定位InputWindowsManager日志

hilog -T InputWindowsManager

4、定位PointerDrawingManager日志

hilog -T PointerDrawingManager

5、整体定位mmi服务的日志

hilog -P `pidof multimodalinput`

3.5 参考社区pr

针对此问题，可以借助社区的pr进行方向性思考，如下pr通过解决多屏同显，多屏异显对multimodalinput做了修改

https://gitee.com/openharmony/multimodalinput_input/pulls/3103/files

对于此笔pr，我们可以发现对于绘制光标的surface接口，可以做如下改变：

surfaceNode_->DetachToDisplay(screenId_);      ---->     Rosen::DisplayManager::GetInstance().RemoveSurfaceNodeFromDisplay(displayId_, surfaceNode_);
surfaceNode_->AttachToDisplay(screenId_);     ---->     Rosen::DisplayManager::GetInstance().AddSurfaceNodeToDisplay(displayId_, surfaceNode_);

可以发现，对于光标，openharmony默认直接使用surfaceNode对象来attach和detach对于的display id。此方法在3588使用hdmi时，并不生效。

而通过使用窗管代码的RemoveSurfaceNodeFromDisplay/AddSurfaceNodeToDisplay的接口，可以正常的将光标的surface绑定到对于的display上。此方法有效。

四、代码改动

根据上述内容，我们可以清楚的知道光标的实现逻辑以及代码的修改点，如下贴出代码的提交改动

BUILD.gn改动

diff --git a/service/BUILD.gn b/service/BUILD.gn
index aa307c7ac..5b908c7d8 100644
--- a/service/BUILD.gn
+++ b/service/BUILD.gn
@@ -164,7 +164,10 @@ ohos_shared_library("libmmi-server") {
external_deps = [ "c_utils:utils" ]
 if (input_feature_pointer_drawing) {
- external_deps += [ "window_manager:libwm" ]
+ external_deps += [
+ "window_manager:libdm",
+ "window_manager:libwm",
+ ]
 }
if (security_component_enable) {
window_manager/include/pointer_drawing_manager.h改动

diff --git a/service/window_manager/include/pointer_drawing_manager.h b/service/window_manager/include/pointer_drawing_manager.h
index 52a0ee255..d111b02c6 100644
--- a/service/window_manager/include/pointer_drawing_manager.h
+++ b/service/window_manager/include/pointer_drawing_manager.h
@@ -105,7 +105,7 @@ private:
 std::list<PidInfo> pidInfos_;
 bool mouseDisplayState_ { false };
 std::unique_ptr<OHOS::Media::PixelMap> userIcon_ { nullptr };
- uint64_t screenId_ { 0 };
+ Rosen::DisplayId displayId_ { 0 };
 std::shared_ptr<Rosen::RSSurfaceNode> surfaceNode_;
 std::shared_ptr<Rosen::RSCanvasNode> canvasNode_;
 int32_t userIconHotSpotX_ { 0 };
window_manager/src/input_windows_manager.cpp改动

diff --git a/service/window_manager/src/input_windows_manager.cpp b/service/window_manager/src/input_windows_manager.cpp
index c31c103d5..77a98401f 100644
--- a/service/window_manager/src/input_windows_manager.cpp
+++ b/service/window_manager/src/input_windows_manager.cpp
@@ -1470,7 +1470,10 @@ int32_t InputWindowsManager::UpdateTargetPointer(std::shared_ptr<PointerEvent> p
 #ifdef OHOS_BUILD_ENABLE_POINTER
 bool InputWindowsManager::IsInsideDisplay(const DisplayInfo& displayInfo, int32_t physicalX, int32_t physicalY)
 {
- return (physicalX >= 0 && physicalX < displayInfo.width) && (physicalY >= 0 && physicalY < displayInfo.height);
+ if (displayInfo.direction == DIRECTION0 || displayInfo.direction == DIRECTION180) {
+ return (physicalX >= 0 && physicalX < displayInfo.width) && (physicalY >= 0 && physicalY < displayInfo.height);
+ }
+ return (physicalX >= 0 && physicalX < displayInfo.height) && (physicalY >= 0 && physicalY < displayInfo.width);
 }
void InputWindowsManager::FindPhysicalDisplay(const DisplayInfo& displayInfo, int32_t& physicalX,
@@ -1490,14 +1493,25 @@ void InputWindowsManager::FindPhysicalDisplay(const DisplayInfo& displayInfo, in
 for (const auto &item : displayGroupInfo_.displaysInfo) {
 int32_t displayMaxX = 0;
 int32_t displayMaxY = 0;
- if (!AddInt32(item.x, item.width, displayMaxX)) {
- MMI_HILOGE("The addition of displayMaxX overflows");
- return;
- }
- if (!AddInt32(item.y, item.height, displayMaxY)) {
- MMI_HILOGE("The addition of displayMaxY overflows");
- return;
- }
+ if (item.direction == DIRECTION0 || item.direction == DIRECTION180) {
+ if (!AddInt32(item.x, item.width, displayMaxX)) {
+ MMI_HILOGE("The addition of displayMaxX overflows");
+ return;
+ }
+ if (!AddInt32(item.y, item.height, displayMaxY)) {
+ MMI_HILOGE("The addition of displayMaxY overflows");
+ return;
+ }
+ } else {
+ if (!AddInt32(item.x, item.height, displayMaxX)) {
+ MMI_HILOGE("The addition of displayMaxX overflows");
+ return;
+ }
+ if (!AddInt32(item.y, item.width, displayMaxY)) {
+ MMI_HILOGE("The addition of displayMaxY overflows");
+ return;
+ }
+ }
 if ((logicalX >= item.x && logicalX < displayMaxX) &&
 (logicalY >= item.y && logicalY < displayMaxY)) {
 physicalX = logicalX - item.x;

window_manager/src/pointer_drawing_manager.cpp改动

diff --git a/service/window_manager/src/pointer_drawing_manager.cpp b/service/window_manager/src/pointer_drawing_manager.cpp
index cc53d196e..e51fb4c63 100644
--- a/service/window_manager/src/pointer_drawing_manager.cpp
+++ b/service/window_manager/src/pointer_drawing_manager.cpp
@@ -19,6 +19,7 @@
 #include "image_source.h"
 #include "image_type.h"
 #include "image_utils.h"
+#include "display_manager.h"
#include "define_multimodal.h"
 #include "input_device_manager.h"
@@ -83,6 +84,12 @@ void PointerDrawingManager::DrawPointer(int32_t displayId, int32_t physicalX, in
 physicalY + displayInfo_.y,
 surfaceNode_->GetStagingProperties().GetBounds().z_,
 surfaceNode_->GetStagingProperties().GetBounds().w_);
+ if (displayId_ != static_cast<Rosen::DisplayId>(displayId)) {
+ // Move pointer to another display. 'Add' before 'Remove' to avoid the surfaceNode_ to be destroyed by dms.
+ Rosen::DisplayManager::GetInstance().AddSurfaceNodeToDisplay(displayId, surfaceNode_);
+ Rosen::DisplayManager::GetInstance().RemoveSurfaceNodeFromDisplay(displayId_, surfaceNode_);
+ displayId_ = static_cast<Rosen::DisplayId>(displayId);
+ }
 Rosen::RSTransaction::FlushImplicitTransaction();
 MMI_HILOGD("Pointer window move success");
 if (lastMouseStyle_.id == mouseStyle) {
@@ -125,7 +132,7 @@ int32_t PointerDrawingManager::InitLayer(const MOUSE_ICON mouseStyle)
 sptr<OHOS::Surface> layer = GetLayer();
 if (layer == nullptr) {
 MMI_HILOGE("Init layer is failed, Layer is nullptr");
- surfaceNode_->DetachToDisplay(screenId_);
+ Rosen::DisplayManager::GetInstance().RemoveSurfaceNodeFromDisplay(displayId_, surfaceNode_);
 surfaceNode_ = nullptr;
 Rosen::RSTransaction::FlushImplicitTransaction();
 MMI_HILOGD("Pointer window destroy success");
@@ -135,7 +142,7 @@ int32_t PointerDrawingManager::InitLayer(const MOUSE_ICON mouseStyle)
 sptr<OHOS::SurfaceBuffer> buffer = GetSurfaceBuffer(layer);
 if (buffer == nullptr || buffer->GetVirAddr() == nullptr) {
 MMI_HILOGE("Init layer is failed, buffer or virAddr is nullptr");
- surfaceNode_->DetachToDisplay(screenId_);
+ Rosen::DisplayManager::GetInstance().RemoveSurfaceNodeFromDisplay(displayId_, surfaceNode_);
 surfaceNode_ = nullptr;
 Rosen::RSTransaction::FlushImplicitTransaction();
 MMI_HILOGD("Pointer window destroy success");
@@ -312,9 +319,10 @@ void PointerDrawingManager::CreatePointerWindow(int32_t displayId, int32_t physi
 surfaceNode_->SetPositionZ(Rosen::RSSurfaceNode::POINTER_WINDOW_POSITION_Z);
 surfaceNode_->SetBounds(physicalX, physicalY, IMAGE_WIDTH, IMAGE_HEIGHT);
 surfaceNode_->SetBackgroundColor(SK_ColorTRANSPARENT);
- screenId_ = static_cast<uint64_t>(displayId);
- std::cout << "ScreenId: " << screenId_ << std::endl;
- surfaceNode_->AttachToDisplay(screenId_);
+
+ displayId_ = static_cast<Rosen::DisplayId>(displayId);
+ MMI_HILOGD("DisplayId: %{public}" PRIu64 "", displayId_);
+ Rosen::DisplayManager::GetInstance().AddSurfaceNodeToDisplay(displayId_, surfaceNode_);
 surfaceNode_->SetRotation(0);
canvasNode_ = Rosen::RSCanvasNode::Create();
@@ -630,7 +638,7 @@ void PointerDrawingManager::OnDisplayInfo(const DisplayGroupInfo& displayGroupIn
 lastPhysicalY_ = displayGroupInfo.displaysInfo[0].height / CALCULATE_MIDDLE;
 MouseEventHdr->OnDisplayLost(displayInfo_.id);
 if (surfaceNode_ != nullptr) {
- surfaceNode_->DetachToDisplay(screenId_);
+ Rosen::DisplayManager::GetInstance().RemoveSurfaceNodeFromDisplay(displayId_, surfaceNode_);
 surfaceNode_ = nullptr;
 Rosen::RSTransaction::FlushImplicitTransaction();
 MMI_HILOGD("Pointer window destroy success");
@@ -681,7 +689,7 @@ void PointerDrawingManager::DrawManager()
 }
 if (!hasPointerDevice_ && surfaceNode_ != nullptr) {
 MMI_HILOGD("Pointer window destroy start");
- surfaceNode_->DetachToDisplay(screenId_);
+ Rosen::DisplayManager::GetInstance().RemoveSurfaceNodeFromDisplay(displayId_, surfaceNode_);
 surfaceNode_ = nullptr;
 Rosen::RSTransaction::FlushImplicitTransaction();
 MMI_HILOGD("Pointer window destroy success");
@@ -885,7 +893,7 @@ void PointerDrawingManager::DrawPointerStyle()
 CALL_DEBUG_ENTER;
 if (hasDisplay_ && hasPointerDevice_) {
 if (surfaceNode_ != nullptr) {
- surfaceNode_->AttachToDisplay(screenId_);
+ Rosen::DisplayManager::GetInstance().AddSurfaceNodeToDisplay(displayId_, surfaceNode_);
 Rosen::RSTransaction::FlushImplicitTransaction();
 }
 PointerStyle pointerStyle;

代码补丁文件：0001-rk3588-fixed-cursor-is-hided.patch

五、总结

至此，openharmony可正常显示光标，其根本原因是适配时，openharmony的surfaceNode实现的attach接口并不生效，可能和底层相关。但参照pr修改为windows manager的api即正常。此问题应属于系统本身问题