HumanBodyPose¶

class agentlego.tools.HumanBodyPose(model='human', device='cuda', toolmeta=None)[源代码]

A tool to extract human body keypoints from an image.

参数:

model (str) – The model name used to inference. Which can be found in the MMPose repository. Defaults to human.
device (str) – The device to load the model. Defaults to ‘cuda’.
toolmeta (None | dict | ToolMeta) – The additional info of the tool. Defaults to None.

默认工具信息¶

名称: HumanBodyPose
描述: This tool can estimate the pose or keypoints of human in an image and draw the human pose image.
输入:
- image (ImageIO)
输出:
- ImageIO: The human pose keypoints image.

Examples¶

Download the demo resource

wget http://download.openmmlab.com/agentlego/human.jpg

Use the tool directly (without agent)

from agentlego.apis import load_tool

# load tool
tool = load_tool('HumanBodyPose', device='cuda')

# apply tool
image = tool('human.jpg')
print(image)

With Lagent

from lagent import ReAct, GPTAPI, ActionExecutor
from agentlego.apis import load_tool

# load tools and build agent
# please set `OPENAI_API_KEY` in your environment variable.
tool = load_tool('HumanBodyPose', device='cuda').to_lagent()
agent = ReAct(GPTAPI(temperature=0.), action_executor=ActionExecutor([tool]))

# agent running with the tool.
img_path = 'human.jpg'
ret = agent.chat(f'Extract pose of the human in the image {img_path}')
for step in ret.inner_steps[1:]:
    print('------')
    print(step['content'])

Set up¶

Before using the tool, please confirm you have installed the related dependencies by the below commands.

pip install -U openmim
pip install git+https://github.com/jin-s13/xtcocoapi
mim install -U mmpose

Reference¶

This tool uses a RTM Pose model in default settings. See the following paper for details.

@misc{jiang2023rtmpose,
      title={RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose},
      author={Tao Jiang and Peng Lu and Li Zhang and Ningsheng Ma and Rui Han and Chengqi Lyu and Yining Li and Kai Chen},
      year={2023},
      eprint={2303.07399},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}