当前位置：首页 > news >正文

OpenObserve API Usage Guide for Log Management

news 来源：原创 2025/5/20 2:04:47

OpenObserve API Usage Guide for Audit Log Management

1. 概述

1.1 目标

本文档旨在详细介绍 OpenObserve 的 API 使用方法，帮助用户通过 API 实现日志管理功能，包括日志摄入、查询、模糊匹配（类似 SQL 的 LIKE）、stream 管理等。文档基于 test stream 的审计日志场景，解决之前遇到的模糊匹配问题。

1.2 OpenObserve API 简介

OpenObserve 是一个开源的 observability 平台，支持日志、指标和追踪数据。API 是与 OpenObserve 交互的主要方式，支持以下功能：

日志摄入：通过 API 批量摄入日志数据。
日志查询：支持 SQL-like 语法查询日志，支持模糊匹配。
Stream 管理：创建、更新、删除 stream，以及管理 stream 设置（如 full_text_search_keys）。
用户和组织管理：管理组织、用户和权限。
告警和仪表盘：配置告警规则和创建仪表盘。

1.3 认证

OpenObserve API 使用 HTTP Basic Authentication，所有请求必须包含 Authorization 头。认证信息是用户 ID 和密码的 Base64 编码。

生成 Authorization 头

用户 ID：admin@admin.com
密码：admin
Base64 编码：echo -n "admin@admin.com:admin" | base64 -> YWRtaW5AYWRtaW4uY29tOmFkbWlu

示例

-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu"

2. API 端点和使用方法

以下是 OpenObserve 的主要 API 端点及其使用方法，基于 OpenObserve 文档。

2.1 日志摄入 API

端点

POST /api/{organization}/{stream}/_json

功能

批量摄入日志数据到指定 stream。OpenObserve 会根据数据自动推断 schema。

请求

Content-Type：application/json
Body：JSON 数组，每个元素是一条日志记录。

示例

摄入 3 条审计日志到 test stream：

curl -X POST \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \-H "Content-Type: application/json" \-d '[{"user": "test_user_create","path": "/test/path","category": "File Management","event": "Create Document","date": "2025-04-28 17:01:00","comment": "Created a new document","ip": "192.168.1.1"},{"user": "admin_user","path": "/admin/path","category": "User Management","event": "Update User","date": "2025-04-28 17:02:00","comment": "Updated user profile","ip": "192.168.1.2"},{"user": "create_team_lead","path": "/team/path","category": "Team Management","event": "Create Team","date": "2025-04-28 17:03:00","comment": "Created a new team","ip": "192.168.1.3"}]' \http://localhost:5080/api/default/test/_json

响应

{"code": 200,"status": [{"name": "test","successful": 3,"failed": 0}]
}

注意事项

如果 test stream 不存在，OpenObserve 会自动创建。
确保字段类型一致（例如 date 应始终为字符串格式）。

2.2 日志查询 API

端点

POST /api/{organization}/{stream}/_search

功能

使用 SQL-like 语法查询日志，支持模糊匹配、聚合等操作。

请求

Content-Type：application/json
Body：
- query.sql：SQL 查询语句。
- query.start_time 和 query.end_time：查询时间范围（微秒）。
- from 和 size：分页参数。

示例 1：基本查询

查询 test stream 的最新 10 条记录：

curl -X POST \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \-H "Content-Type: application/json" \-d '{"query": {"sql": "SELECT * FROM test","start_time": 0,"end_time": 999999999999999,"from": 0,"size": 10}}' \http://localhost:5080/api/default/test/_search

响应

{"took": 10,"hits": {"total": 3,"hits": [{"user": "test_user_create","path": "/test/path","category": "File Management","event": "Create Document","date": "2025-04-28 17:01:00","comment": "Created a new document","ip": "192.168.1.1"},{"user": "admin_user","path": "/admin/path","category": "User Management","event": "Update User","date": "2025-04-28 17:02:00","comment": "Updated user profile","ip": "192.168.1.2"},{"user": "create_team_lead","path": "/team/path","category": "Team Management","event": "Create Team","date": "2025-04-28 17:03:00","comment": "Created a new team","ip": "192.168.1.3"}]}
}

示例 2：模糊匹配查询

使用 str_match 实现模糊匹配（类似 LIKE '%create%'），查找 user 字段包含 “create” 的记录：

curl -X POST \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \-H "Content-Type: application/json" \-d '{"query": {"sql": "SELECT * FROM test WHERE str_match(user, '\''create'\'')","start_time": 0,"end_time": 999999999999999,"from": 0,"size": 10}}' \http://localhost:5080/api/default/test/_search

响应

{"took": 10,"hits": {"total": 2,"hits": [{"user": "test_user_create","path": "/test/path","category": "File Management","event": "Create Document","date": "2025-04-28 17:01:00","comment": "Created a new document","ip": "192.168.1.1"},{"user": "create_team_lead","path": "/team/path","category": "Team Management","event": "Create Team","date": "2025-04-28 17:03:00","comment": "Created a new team","ip": "192.168.1.3"}]}
}

示例 3：全局全文搜索

使用 match_all 在所有字段中查找 “create”：

curl -X POST \-H "Authorization: Basic YWRtaW5AYWRmiW4uY29tOmFkbWlu" \-H "Content-Type: application/json" \-d '{"query": {"sql": "SELECT * FROM test WHERE match_all('\''create'\'')","start_time": 0,"end_time": 999999999999999,"from": 0,"size": 10}}' \http://localhost:5080/api/default/test/_search

响应

{"took": 10,"hits": {"total": 3,"hits": [{"user": "test_user_create","path": "/test/path","category": "File Management","event": "Create Document","date": "2025-04-28 17:01:00","comment": "Created a new document","ip": "192.168.1.1"},{"user": "admin_user","path": "/admin/path","category": "User Management","event": "Update User","date": "2025-04-28 17:02:00","comment": "Updated user profile","ip": "192.168.1.2"},{"user": "create_team_lead","path": "/team/path","category": "Team Management","event": "Create Team","date": "2025-04-28 17:03:00","comment": "Created a new team","ip": "192.168.1.3"}]}
}

注意事项

str_match 不需要字段在 full_text_search_keys 中，但如果启用全文搜索，查询效率可能更高。
match_all 依赖 full_text_search_keys，否则可能无法匹配。

2.3 Stream 管理 API

端点 1：列出所有 Streams

GET /api/{organization}/streams

功能

列出组织中的所有 stream，包括 schema 和设置。

示例

curl -X GET \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \http://localhost:5080/api/default/streams

响应

[{"name": "test","storage_type": "disk","stream_type": "logs","stats": {"doc_num": 3,"storage_size": 0.0003},"schema": [{"name": "_timestamp", "type": "Int64"},{"name": "category", "type": "Utf8"},{"name": "comment", "type": "Utf8"},{"name": "date", "type": "Utf8"},{"name": "event", "type": "Utf8"},{"name": "ip", "type": "Utf8"},{"name": "path", "type": "Utf8"},{"name": "user", "type": "Utf8"}],"settings": {"full_text_search_keys": [],"data_retention": 0}}
]

端点 2：更新 Stream 设置

PUT /api/{organization}/streams/{stream}/settings

功能

更新 stream 的设置，例如 full_text_search_keys。

示例

尝试启用 user 字段的全文搜索：

curl -X PUT \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \-H "Content-Type: application/json" \-d '{"full_text_search_keys": ["user"]}' \http://localhost:5080/api/default/streams/test/settings

响应

{"code": 200,"message": "Settings updated successfully for stream test"
}

注意事项

之前尝试更新 full_text_search_keys 失败（Json deserialize error），建议通过 UI 手动设置，或者联系 OpenObserve 社区解决。

端点 3：删除 Stream

DELETE /api/{organization}/streams/{stream}

功能

删除指定 stream 及其数据。

示例

curl -X DELETE \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \http://localhost:5080/api/default/streams/test

响应

{"code": 200,"message": "Stream test deleted successfully"
}

2.4 告警 API

端点

POST /api/{organization}/alerts

功能

创建告警规则，例如当 event 字段包含 “Error” 时触发告警。

示例

创建告警，当 event 包含 “Error” 时触发：

curl -X POST \-H "Authorization: Basic YWRtaW5AYWRtaW4uY29tOmFkbWlu" \-H "Content-Type: application/json" \-d '{"name": "error_alert","stream": "test","query": "SELECT * FROM test WHERE str_match(event, '\''Error'\'')","condition": "num > 0","threshold": 1,"frequency": "5m","destination": "email","email": "admin@example.com"}' \http://localhost:5080/api/default/alerts

响应

{"code": 200,"message": "Alert created successfully"
}

3. 总结

OpenObserve 的 API 提供了强大的日志管理功能，支持摄入、查询、模糊匹配和 stream 管理。通过 str_match 可以实现类似 LIKE 的模糊匹配，满足审计日志场景的需求。建议进一步优化 full_text_search_keys 设置，并关注 OpenObserve 的版本更新以获取更多功能支持（如 REGEXP 或 LIKE）。

OpenObserve API Usage Guide for Audit Log Management

1. 概述

1.1 目标

1.2 OpenObserve API 简介

1.3 认证

生成 Authorization 头

示例

2. API 端点和使用方法

2.1 日志摄入 API

端点

功能

请求

示例

响应

注意事项

2.2 日志查询 API

端点

功能

请求

示例 1：基本查询

响应

示例 2：模糊匹配查询

响应

示例 3：全局全文搜索

响应

注意事项

2.3 Stream 管理 API

端点 1：列出所有 Streams

功能

示例

响应

端点 2：更新 Stream 设置

功能

示例

响应

注意事项

端点 3：删除 Stream

功能

示例

响应

2.4 告警 API

端点

功能

示例

响应

3. 总结

相关文章：