diff --git a/content/cn/docs/clients/gremlin-console.md b/content/cn/docs/clients/gremlin-console.md index e91bcc9dc1..3129a6fb1f 100644 --- a/content/cn/docs/clients/gremlin-console.md +++ b/content/cn/docs/clients/gremlin-console.md @@ -4,25 +4,25 @@ linkTitle: "Gremlin Console" weight: 3 --- -Gremlin-Console 是由 Tinkerpop 自己开发的一个交互式客户端，用户可以使用该客户端对 Graph 做各种操作，主要有两种使用模式： +Gremlin-Console 是 TinkerPop 的交互式客户端，支持离线操作图，也可以连接 Server 提交查询。 - 单机离线调用模式 - Client/Server 请求模式 -**注：Gremlin-Console 只是便于用户快速上手体验，不建议在生产环境中使用。** +Gremlin-Console 适合学习和调试。生产应用建议使用对应语言的客户端。 ### 1 单机离线调用模式 -由于 lib 目录下已经包含了 HugeCore 的 jar 包，且 HugeGraph-Server 已经作为插件注册到 Gremlin-Console 中，用户可以直接写 Groovy 脚本调用 HugeGraph-Core 的代码，然后交由 Gremlin-Console 内的解析引擎执行，就能在**不启动** Server 的情况下操作图。 +Server 发行包内的 Console 已包含 HugeGraph-Core JAR 和插件，可以直接执行 Groovy 脚本操作图，无需启动 Server。 -这里提供一个示例，首先修改 `hugegraph.properties` 配置使用 Memory 后端 (使用其他后端可能会出现一些初始化问题)： +先在 `hugegraph.properties` 中配置 Memory 后端。其他后端可能需要额外初始化。 ```properties backend=memory serializer=text ``` -然后输入下述命令： +然后运行示例脚本。 ```bash > ./bin/gremlin-console.sh -- -i scripts/example.groovy @@ -45,7 +45,7 @@ gremlin> 其中 [`example.groovy`](https://github.com/apache/hugegraph/blob/master/hugegraph-server/hugegraph-dist/src/assembly/static/scripts/example.groovy) 是 scripts 目录下的一个示例脚本，该脚本插入了一些数据，并在最后查询图中顶点和边的数量。 -此时还可以继续输入 Gremlin 语句对图进行操作： +脚本执行后，可以继续输入 Gremlin 语句。 ```groovy gremlin> g.V() @@ -69,73 +69,78 @@ gremlin> ### 2 Client/Server 请求模式 -Gremlin-Console 通过 WebSocket 连接 HugeGraph-Server。默认配置使用 `WsAndHttpChannelizer`，可同时处理 WebSocket 和 HTTP 请求，无需切换 Channelizer。 +Gremlin-Console 通过 WebSocket 连接 HugeGraph-Server。使用与 Server 同版本发行包中的 Console 和 `conf/remote.yaml`， +保留文件中的序列化器配置，只修改地址，以及启用鉴权时需要的凭据。旧版配置可能引用已经不存在的序列化器类。 -```yaml -# vim conf/gremlin-server.yaml -# ...... -channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer -# ...... -``` +1. 准备连接配置并启动 Server。 -确认 `host`、`port` 与 `remote.yaml` 一致，然后按照[步骤](/cn/docs/quickstart/hugegraph/hugegraph-server/)启动 HugeGraph-Server。 + 检查 `conf/gremlin-server.yaml` 中的 `host`、`port`。默认的 `WsAndHttpChannelizer` 同时处理 WebSocket 和 HTTP 请求，无需修改。 + `conf/remote.yaml` 中的 `hosts` 应填写 Console 能访问的 Server 地址，`port` 应与 Gremlin 端口一致（默认为 `8182`，不是 REST 端口 `8080`）。 + 按照[快速上手](/cn/docs/quickstart/hugegraph/hugegraph-server/)启动 Server。 -下面进入 Gremlin-Console： + 当前 master 配置使用 GraphBinary V1 和 HugeGraph 类型注册器。 -```bash -> ./bin/gremlin-console.sh + ```yaml + hosts: [localhost] + port: 8182 + serializer: + className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1 + config: + serializeResultToString: false + builder: org.apache.hugegraph.io.HugeGraphTypeSerializerRegistryBuilder + ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry] + ``` - \,,,/ - (o o) ------oOOo-(3)-oOOo----- -plugin activated: HugeGraph -plugin activated: tinkerpop.server -plugin activated: tinkerpop.utilities -plugin activated: tinkerpop.tinkergraph -``` + 如果 Server 开启了鉴权，在同一个文件中填写实际凭据。 -连接 Server，需在配置文件中指定连接参数，在 conf 目录下有一个默认的 `remote.yaml`： - -```yaml -# cat conf/remote.yaml -hosts: [localhost] -port: 8182 -serializer: { - className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, - config: { - serializeResultToString: false, - ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry] - } -} -``` + ```yaml + username: admin + password: "replace-with-your-password" + ``` -如果 Server 开启了鉴权模式，需要在同一个文件中补上凭据： + 随包的 `remote-objects.yaml` 和 `gremlin-driver-settings.yaml` 使用相同的序列化器配置；使用这些文件时也要保持一致。 -```yaml -username: admin -password: pa -``` +1. 从 Server 发行包目录启动 Console，并创建带 traversal 别名的客户端。 -`conf` 目录下还提供了 `remote-objects.yaml` 和 `gremlin-driver-settings.yaml`，它们使用相同的 host、port 和序列化器配置。 + ```bash + ./bin/gremlin-console.sh + ``` -```groovy -gremlin> :remote connect tinkerpop.server conf/remote.yaml -==>Configured localhost/127.0.0.1:8182 -``` + Server 端将默认图空间下的图绑定为 `DEFAULT-hugegraph`，其 traversal source 绑定为 `__g_DEFAULT-hugegraph`。 + 由于连字符 `-` 在 Groovy 中不能直接用作变量名，通过 Java Driver 连接时，使用 `.alias(['g': '__g_DEFAULT-hugegraph'])` 将其映射为 `g`。 + + ```groovy + import org.apache.tinkerpop.gremlin.driver.Cluster + cluster = Cluster.open('conf/remote.yaml') + client = cluster.connect().alias(['g': '__g_DEFAULT-hugegraph']) + ``` + +1. 通过客户端提交查询。 + + ```groovy + client.submit('g.V().count()').all().get()[0].object + client.submit('g.V().toList().size()').all().get()[0].object + ``` + + 若启动 Server 时预加载了示例图，两条查询均返回 `6`（其他图返回其实际顶点数）。 + 通过 Driver 的 `client.submit()` 提交脚本时，无需在语句前加 `:>` 前缀。 + +1. 使用结束后关闭客户端和集群连接。 + + ```groovy + client.close() + cluster.close() + ``` +{.steps} -Server 端的图以图空间限定名绑定，因此图空间 `DEFAULT` 下的图 `hugegraph` 绑定名为 `DEFAULT-hugegraph`，其 traversal source 绑定名为 `__g_DEFAULT-hugegraph`。裸写 `hugegraph` 在 Server 端无法解析，而 `DEFAULT-hugegraph` 又不是合法的 Groovy 标识符，所以远程脚本需要通过别名访问 traversal source。如果启动 HugeGraph-Server 时预加载了示例图，查询方式如下： +此外，也可以使用 Gremlin Console 内置的 `:remote` 命令连接： ```groovy -gremlin> import org.apache.tinkerpop.gremlin.driver.Cluster -gremlin> cluster = Cluster.open('conf/remote.yaml') -gremlin> client = cluster.connect().alias(['g': '__g_DEFAULT-hugegraph']) -gremlin> client.submit('g.V().count()').all().get()[0].object -==>6 -gremlin> client.submit('g.V().toList().size()').all().get()[0].object -==>6 -gremlin> client.close(); cluster.close() +:remote connect tinkerpop.server conf/remote.yaml +// 通过 Console 远程连接提交脚本时，前缀使用 :> +:remote close ``` -> 注意：在 Client/Server 模式下，所有和 Server 有关的操作都要加上 `:> `，如果不加，表示在 console 本地操作。`:> ` 发送的脚本不带别名，因此只能使用 Server 自身已绑定的名称。 +使用 `:remote` 连接时，以 `:>` 开头的语句会被发送至远端 Server 执行，普通语句则在 Console 本地执行。 -更多关于 Gremlin-Console 的使用，请参考 [Tinkerpop 官网](http://tinkerpop.apache.org/docs/current/reference/)。 +更多用法见 [TinkerPop Gremlin Console 参考](https://tinkerpop.apache.org/docs/current/reference/#gremlin-console)。 diff --git a/content/cn/docs/config/config-authentication.md b/content/cn/docs/config/config-authentication.md index 36f8de2280..ef125e0981 100644 --- a/content/cn/docs/config/config-authentication.md +++ b/content/cn/docs/config/config-authentication.md @@ -10,13 +10,11 @@ search_boost: 1.7 HugeGraph 内置 `StandardAuthenticator`，支持多用户认证和基于“用户、用户组、操作、资源”的权限控制。 -`StandardAuthenticator` 模式的几个核心设计： -- 初始化时创建超级管理员 (`admin`) 用户，后续通过超级管理员创建其它用户，新创建的用户被分配足够权限后，可以创建或管理更多的用户 -- 支持动态创建用户、用户组、资源，支持动态分配或取消权限 -- 用户可以属于一个或多个用户组，每个用户组可以拥有对任意个资源的操作权限，操作类型包括：读、写、删除、执行等种类 -- "资源" 描述了图数据库中的数据，比如符合某一类条件的顶点，每一个资源包括 `type`、`label`、`properties`三个要素，共有 18 种类型、任意 label、任意 properties 可组合形成的资源，一个资源的内部条件是且关系，多个资源之间的条件是或关系 +初始化会创建超级管理员 `admin`。管理员可创建用户、用户组及资源，并分配或撤销权限。用户通过所属组获得读、写、删除和执行等权限。 -举例说明： +资源用 `type`、`label` 和 `properties` 描述可访问的数据。同一资源中的条件需同时满足，多个资源之间满足其一即可。 + +下面的关系表示用户仅可读取北京地区的数据。 ```java // 场景：某用户只有北京地区的数据读取权限 @@ -30,11 +28,13 @@ HugeGraph 目前默认**未启用**用户认证功能，需通过修改配置文 > [!WARNING] > **生产环境必须启用鉴权** > -> HugeGraph 默认不启用用户认证。生产环境必须启用认证与授权、设置非默认强管理员密码、启用并维护 Server IP 白名单，并按最小权限为用户授权；不得将 Gremlin、Cypher 等查询接口直接暴露到公网。Server 标准配置会将鉴权代理记录写入 `audit-*.log`，部署时必须保留并限制这些日志的读取权限。`auth.audit_log_rate` 只限制每用户的日志输出速率，不是审计日志的启停开关。 +> HugeGraph 默认不启用用户认证。生产环境必须启用认证与授权、设置非默认强管理员密码、启用并维护 Server IP 白名单，并按最小权限为用户授权；不得将 Gremlin、Cypher 等查询接口直接暴露到公网。Server 标准配置会将鉴权代理记录写入 +`audit-*.log`，部署时必须保留并限制这些日志的读取权限。`auth.audit_log_rate` 只限制每用户的日志输出速率，不是审计日志的启停开关。 `StandardAuthenticator` 支持多用户认证和细粒度权限控制。也可以实现 `HugeAuthenticator` 接口来接入已有的用户系统。 -用户认证使用 [HTTP Basic Authentication](https://zh.wikipedia.org/wiki/HTTP%E5%9F%BA%E6%9C%AC%E8%AE%A4%E8%AF%81)。`Basic` 后面的值是 `用户名:密码` 的 Base64 编码。使用 curl 时可直接通过 `-u` 传入凭据： +用户认证使用 [HTTP Basic Authentication](https://zh.wikipedia.org/wiki/HTTP%E5%9F%BA%E6%9C%AC%E8%AE%A4%E8%AF%81)。`Basic` 后面的值是 `用户名:密码` 的 Base64 编码。使用 curl 时可直接通过 +`-u` 传入凭据。 ```bash curl -u 'admin:' \ @@ -52,7 +52,7 @@ curl -u 'admin:' \ > HugeGraph-Server 1.5.0 之前的版本在鉴权模式下存在 JWT 相关安全隐患。使用受影响版本时必须升级，或按该版本要求修改 JWT token 的 `secretKey`。 `auth.token_secret` 由认证数据图的配置读取。默认认证图名为 `hugegraph`，因此配置文件通常是 `conf/graphs/hugegraph.properties`；如果修改了 `auth.graph_store`，应改为对应图的 properties 文件。 -源码默认会生成 32 个随机字节并以 Base64 编码，但该值不会写回配置文件；重启后或多个 Server 节点之间无法依赖各自生成的默认值保持一致。若需要令已有 token 在重启后继续有效，或让多个节点验证同一 token，请为认证图显式配置同一个强随机密钥： +核心配置默认会生成 32 个随机字节并以 Base64 编码，该默认值不会自行写回配置文件；重启后或多个 Server 节点之间无法依赖各自生成的默认值保持一致。若需要令已有 token 在重启后继续有效，或让多个节点验证同一 token，请为认证图显式配置同一个强随机密钥。 Server 将配置字符串按 UTF-8 编码后交给 JJWT 的 HS256 签名器，密钥至少需要 32 字节；这是字节长度要求，不代表任意 32 字符串都具有足够随机性。下面的命令在本机生成 32 个随机字节并以 Base64 编码。 @@ -61,20 +61,20 @@ Server 将配置字符串按 UTF-8 编码后交给 JJWT 的 HS256 签名器， auth.token_secret=<本机生成的密钥> ``` -可在本机终端生成密钥，再通过安全方式填入认证图配置；不要把实际密钥提交到代码库或公开日志： +可在本机终端生成密钥，再通过安全方式填入认证图配置；不要把实际密钥提交到代码库或公开日志。 ```bash openssl rand -base64 32 ``` -由于默认值在每次启动时随机生成，当 token 需要在重启后继续有效、或者需要被多个服务节点接受时，必须显式配置该项。token 的有效期由 `auth.token_expire` 决定，默认为 86400 秒。 +token 的有效期由 `auth.token_expire` 决定，默认为 86400 秒。 `auth.token_expire` 和 `auth.token_secret` 都属于 `auth.graph_store` 指定的认证数据图配置；`rest-server.properties` 中的同名项不会覆盖认证图配置。 #### StandardAuthenticator 模式 -`StandardAuthenticator`模式是通过在数据库后端存储用户信息来支持用户认证和权限控制，该实现基于数据库存储的用户的名称与密码进行认证（密码已被加密），基于用户的角色来细粒度控制用户权限。下面是具体的配置流程（重启服务生效）： +`StandardAuthenticator` 在后端存储用户及密码哈希，并按用户角色控制访问。修改以下三处配置后，按[启动顺序](#基于鉴权模式启动)初始化并重启服务。 -在配置文件`gremlin-server.yaml`中配置`authenticator`及其`rest-server`文件路径： +在 `conf/gremlin-server.yaml` 中配置认证器及 REST 配置路径。 ```yaml authentication: { @@ -84,7 +84,7 @@ authentication: { } ``` -在 `rest-server.properties` 中配置认证器和权限数据存储图： +在 `conf/rest-server.properties` 中配置认证器和权限数据存储图。 ```properties auth.authenticator=org.apache.hugegraph.auth.StandardAuthenticator @@ -98,9 +98,9 @@ auth.graph_store=hugegraph #auth.remote_url=127.0.0.1:8899,127.0.0.1:8898,127.0.0.1:8897 ``` -其中，`graph_store`配置项是指使用哪一个图来存储用户信息，如果存在多个图的话，选取任意一个均可。 +`auth.graph_store` 指定保存认证数据的图，名称必须与 `conf/graphs/` 中的图配置对应。 -在配置文件`hugegraph{n}.properties`中配置`gremlin.graph`信息： +在每个受保护图的配置文件 `conf/graphs/.properties` 中设置 `gremlin.graph`。 ```properties gremlin.graph=org.apache.hugegraph.auth.HugeFactoryAuthProxy @@ -110,62 +110,152 @@ gremlin.graph=org.apache.hugegraph.auth.HugeFactoryAuthProxy ### 自定义用户认证系统 -如果需要支持更加灵活的用户系统，可自定义 authenticator 进行扩展，自定义 authenticator 实现接口`org.apache.hugegraph.auth.HugeAuthenticator`即可，然后修改配置文件中`authenticator`配置项指向该实现。 +实现 `org.apache.hugegraph.auth.HugeAuthenticator` 接口，再将两处 `authenticator` 配置指向该实现，即可接入已有用户系统。 ### 基于鉴权模式启动 -首次执行 `init-store.sh` 时，如果尚未创建 `admin` 用户，命令会要求输入管理员密码。对于已经初始化的持久化后端，`init-store.sh` 会补充认证所需的系统信息，无需删除原有图数据。 +下面的顺序适用于本地持久化后端，例如 RocksDB。先完成上面的配置，再初始化认证数据。 +HStore/PD 部署的管理员由 PD 启动路径创建，须按该部署的配置设置 `auth.admin_pa`，不要照搬本地后端的交互式初始化流程。 -```bash -# stop the hugeGraph firstly -bin/stop-hugegraph.sh +1. **停止 Server。** -# 初始化认证系统信息；已有后端数据会被保留 -bin/init-store.sh + ```bash + bin/stop-hugegraph.sh + ``` -# start hugeGraph again -bin/start-hugegraph.sh +1. **初始化认证数据。** -``` + ```bash + bin/init-store.sh + ``` + + 首次创建 `admin` 时，按提示输入强密码。已有图数据会保留；已有 `admin` 时不会重新设置其密码。 + 此命令补充后端与认证系统信息，不需要删除原有存储目录。 + +1. **启动并检查认证。** + + ```bash + bin/start-hugegraph.sh + curl -u admin \ + http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels + ``` + + curl 会提示输入密码。正确密码应返回 HTTP 200；不带凭据或使用错误密码应返回 HTTP 401。 + 启动失败时先检查日志和认证图配置，不要通过清空数据来重试。 +{.steps} ### 使用 Docker 时开启鉴权模式 -对于镜像 `hugegraph/hugegraph` 大于等于 `1.2.0` 的版本，我们可以在启动 `docker` 镜像的同时开启鉴权模式 +新容器在 Docker run 和 Compose 中选择一种方式。已经运行过的容器直接阅读[既有容器](#3-进入容器后重新开启鉴权模式)。 -具体做法如下： +使用 Docker 部署时，官方镜像支持通过环境变量直接开启鉴权模式（适用于官方 Server 镜像 `/hugegraph-server` 布局）。 +将占位符替换为实际镜像标签或 digest： -#### 1. 采用 docker run +```bash +export HUGEGRAPH_IMAGE='' +read -r -s -p 'Admin password: ' PASSWORD +printf '\n' +export PASSWORD +``` + +提示输入管理员密码后执行启动命令。为保障凭据安全，请避免将密码以明文形式提交到共享 Compose 文件中。 +JWT 密钥仍按[上面的配置](#配置用户认证)持久保存到认证图配置；多节点必须使用同一个值。 -在 `docker run` 中添加环境变量 `PASSWORD=xxx`（密码可以自由设置）即可开启鉴权模式：： +#### Docker run {#1-采用-docker-run} + +创建一个新容器，并仅向本机开放 REST 端口。 ```bash -docker run -itd -e PASSWORD=xxx --name=server -p 8080:8080 hugegraph/hugegraph:1.7.0 +docker run -d --name server \ + -p 127.0.0.1:8080:8080 \ + -e PASSWORD \ + "$HUGEGRAPH_IMAGE" +unset PASSWORD ``` -#### 2. 采用 docker-compose +入口脚本完成认证配置和首次初始化后启动 Server。接着阅读[检查认证结果](#检查认证结果)。 + +#### Docker Compose {#2-采用-docker-compose} -使用 `docker-compose` 在环境变量中设置 `PASSWORD=xxx`即可 +将下面的内容保存为 `compose.yaml`。必填变量缺失时，Compose 会报错，避免意外启动无认证服务。 +在同一终端中启动服务。 ```yaml -version: '3' services: server: - image: hugegraph/hugegraph:1.7.0 + image: ${HUGEGRAPH_IMAGE:?Set HUGEGRAPH_IMAGE} container_name: server ports: - - 8080:8080 + - "127.0.0.1:8080:8080" environment: - - PASSWORD=xxx + PASSWORD: ${PASSWORD:?Set PASSWORD} +``` + +```bash +docker compose up -d +unset PASSWORD ``` -#### 3. 进入容器后重新开启鉴权模式 +以后重新创建容器前，需要重新设置这些变量。保留已有数据卷，并按部署要求持久保存配置。 +接着阅读[检查认证结果](#检查认证结果)。 + +#### 既有容器 {#3-进入容器后重新开启鉴权模式} + +此路径适用于原先未开启认证、使用本地持久化后端的官方 Server 容器。 +先备份数据，并确认配置、后端数据及 `conf-bak` 均位于容器挂载的持久卷中。 +官方镜像将 `/hugegraph-server` 声明为卷；自定义镜像应先检查挂载位置。 +HStore/PD 或外部认证服务使用各自的管理员初始化流程。 + +1. **停止原容器，使用同一镜像和数据卷打开临时维护容器。** + + ```bash + SERVER_IMAGE=$(docker inspect --format '{{.Image}}' server) + docker stop server + docker run --rm -it --volumes-from server \ + --entrypoint bash "$SERVER_IMAGE" + ``` + + 原容器须保持停止，防止两个 JVM 同时打开本地数据库。`--entrypoint bash` 使维护容器只打开终端。 + +1. **在维护容器内修改配置并初始化。** + + ```bash + cd /hugegraph-server + bin/enable-auth.sh + ``` + + `enable-auth.sh` 首次运行时将原配置保存到 `conf-bak`，然后修改默认 `hugegraph` 图和认证配置。 + 如果 `conf-bak` 已存在，脚本会跳过修改，须按[StandardAuthenticator 模式](#standardauthenticator-模式)检查三处配置。 + 使用其他认证图时也应手动配置对应文件。 + 按[上面的配置](#配置用户认证)将强随机 JWT 密钥写入认证图配置，再运行初始化命令。 + + ```bash + bin/init-store.sh + exit + ``` + + 按提示输入管理员密码，确认初始化成功后退出。 + +1. **在宿主机启动原容器。** + + ```bash + docker start server + ``` + + 不需要删除 `docker/init_complete` 标记。后续 `PASSWORD` 环境变量的变化不会修改已存储的管理员密码。 +{.steps} + +#### 检查认证结果 -首先进入容器： +等待 Server 启动后，在宿主机执行以下请求验证。`-u admin` 会交互式提示输入密码： ```bash -docker exec -it server bash -# 用于快速修改配置, 修改前的文件被保存在conf-bak文件夹下 -bin/enable-auth.sh +curl -sS -o /dev/null -w '%{http_code}\n' \ + http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels +curl -sS -o /dev/null -w '%{http_code}\n' -u admin \ + http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels ``` -之后参照 [基于鉴权模式启动](#基于鉴权模式启动) 即可 +未携带凭据的请求预期返回 `401 Unauthorized`，输入正确密码的请求返回 `200 OK`（若输入错误密码则同样返回 `401`）。 +注：当前主线版本 URL 包含 `/graphspaces/DEFAULT`，1.7.0 等历史版本请省略 graphspace 前缀（直接使用 `/graphs/hugegraph/...`）。 +若未提供凭据仍返回 `200`，说明鉴权未成功启用，请检查配置并重启 Server。 diff --git a/content/cn/docs/guides/backup-restore.md b/content/cn/docs/guides/backup-restore.md index 49d6b298cb..256409d11f 100644 --- a/content/cn/docs/guides/backup-restore.md +++ b/content/cn/docs/guides/backup-restore.md @@ -1,108 +1,128 @@ --- -title: "Backup Restore" +title: "备份与恢复" linkTitle: "备份 & 恢复" weight: 5 --- ## 描述 -Backup 和 Restore 是备份图和恢复图的功能。备份和恢复的数据包括元数据（schema）和图数据（vertex 和 edge）。 +Backup 将一张图的 Schema、顶点和边导出为 JSON 数据，Restore 将这些数据导入目标图。 +它们是 [HugeGraph-Tools](/cn/docs/quickstart/toolchain/hugegraph-tools) 的命令，适合图迁移、升级后的恢复及图合并。 #### Backup -将 HugeGraph 系统中的一张图的元数据和图数据以 JSON 格式导出。 +备份可在任意图模式下执行。建议在导出前暂停源图的写入操作，以确保 Schema、顶点和边的数据一致性（分批读取不提供全局事务快照）。 +注意：`backup -t all` 仅导出图的 Schema 和图数据，不包含系统配置文件与用户权限。 #### Restore -将 Backup 导出的JSON格式的数据，重新导入到 HugeGraph 系统中的一个图中。 +恢复数据前，需先根据目标图的状态切换恢复模式。日常读写处于 `NONE` 模式；注意：`LOADING` 模式不能用于 `restore`。 -Restore 有两种模式： +| 目标 | 模式 | Schema ID | 顶点和边 ID | +|---|---|---|---| +| 恢复到没有 Schema 和数据的新图 | `RESTORING` | 使用备份中的 ID | 使用备份中的 ID | +| 合并到已有图 | `MERGING` | 使用目标图的 Schema ID，必要时新建 | 部分 ID 随目标 Schema 重算，见下文 | -- Restoring 模式，将 Backup 导出的元数据和图数据原封不动的恢复到 HugeGraph 系统中。可用于图的备份和恢复，一般目标图是新图（没有元数据和图数据）。比如： - - 系统升级，先备份图，然后升级系统，最后将图恢复到新的系统中 - - 图迁移，从一个 HugeGraph 系统中，使用 Backup 功能将图导出，然后使用 Restore 功能将图导入另一个 HugeGraph 系统中 -- Merging 模式，将 Backup 导出的元数据和图数据导入到另一个已经存在元数据或者图数据的图中，过程中元数据的 ID 可能发生改变，顶点和边的 ID 也会发生相应变化。 - - 可用于合并图 +`MERGING` 中，`PRIMARY_KEY` 顶点 ID 按目标 label ID 重新生成，关联边的端点与 ID 随之调整。 +`AUTOMATIC` 和自定义顶点 ID 保留。合并会复用同名 Schema，先检查双方的定义是否兼容，以及保留的顶点 ID 是否冲突。 +它不提供任意 Schema 冲突或数据冲突的自动解决策略。先在独立目标图验证，再合并正式数据。 ## 使用方法 -可以使用[hugegraph-tools](/cn/docs/quickstart/toolchain/hugegraph-tools)进行图的备份和恢复。 +准备与 Server 版本匹配的 [HugeGraph-Tools](/cn/docs/quickstart/toolchain/hugegraph-tools)，在 Tools 根目录执行以下命令。 +本例从 `8080` 的源图恢复到 `8081` 的空目标图。两个 Server 和目标图应已创建并启动；可将变量改成实际地址及图名。 + +```bash +SOURCE_URL=http://127.0.0.1:8080 +TARGET_URL=http://127.0.0.1:8081 +SOURCE_GRAPH=hugegraph +TARGET_GRAPH=hugegraph +BACKUP_DIR="$PWD/data/hugegraph-backup" +``` + +启用认证时，在每条 Tools 命令的子命令前加入 `--user --password `，并使用具备对应图读写和模式管理权限的账号。 +详细参数及凭据使用方式见 [Tools 参考](/cn/docs/quickstart/toolchain/hugegraph-tools)。 #### Backup +暂停源图写入后，备份全部 Schema、顶点和边。 + ```bash -bin/hugegraph backup -t all -d data +bin/hugegraph --url "$SOURCE_URL" --graph "$SOURCE_GRAPH" \ + backup -t all -d "$BACKUP_DIR" ``` -该命令将 http://127.0.0.1:8080（默认 --url）的 hugegraph 图的全部元数据和图数据备份到data目录下。 - -> Backup 在任意图模式下都可以正常工作，它不会检查图模式 +显式指定的 `-d` 就是备份目录，不会自动附加图名。为每次备份使用独立目录，避免混入旧文件。 +保留完整目录，包括压缩文件；Restore 可读取 Tools 支持的 JSON 备份格式。 #### Restore -Restore 有两种模式： RESTORING 和 MERGING，恢复之前首先要根据需要设置图模式，图处于其他模式时 restore 命令会失败。 +以下流程使用 `RESTORING`。合并时改为 `MERGING`，并先完成上面的兼容性检查。 +操作期间暂停目标图业务写入。 -##### 步骤1：查看并设置图模式 -```bash -bin/hugegraph graph-mode-get -``` +1. **查看并记录原模式，再切换。** -该命令用于查看当前图模式，包括：NONE、RESTORING、MERGING、LOADING。 + ```bash + bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" graph-mode-get + ORIGINAL_MODE=NONE + bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" \ + graph-mode-set -m RESTORING + ``` -```bash -bin/hugegraph graph-mode-set -m RESTORING -``` + 正常业务图应处于 `NONE`。将查询返回的值填入 `ORIGINAL_MODE`，例如下方的 `NONE`。若另一项维护正在使用 `RESTORING`、`MERGING` 或 `LOADING`，先协调该任务，不要覆盖其模式。 -该命令用于设置图模式，Restore 之前可以设置成 RESTORING 或者 MERGING 模式，例子中设置成 RESTORING。 +1. **恢复数据。** -##### 步骤2：Restore 数据 + ```bash + bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" \ + restore -t all -d "$BACKUP_DIR" + ``` -```bash -bin/hugegraph restore -t all -d data -``` + 检查命令输出与各类型的导入结果。若发生错误，请先保留备份目录与日志，并检查目标图的数据状态。 + Restore 不会自动回滚已写入的数据，切勿在部分恢复的图上盲目重跑。 + +1. **还原原模式。** + + 使用第一步记录的 `ORIGINAL_MODE` 还原模式。 + + ```bash + bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" graph-mode-set -m "$ORIGINAL_MODE" + bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" graph-mode-get + ``` -该命令将data目录下的全部元数据和图数据重新导入到 http://127.0.0.1:8080 的 hugegraph 图中。 + 无论恢复成功还是失败，最后都必须将图模式还原为原有模式（通常为 `NONE`）。 + 若恢复失败，先保留现场并排查日志，待重新准备目标图后再恢复业务写入。 +{.steps} -##### 步骤3：恢复图模式 +还原模式后，对比两端的顶点数和边数。 ```bash -bin/hugegraph graph-mode-set -m NONE +bin/hugegraph --url "$SOURCE_URL" --graph "$SOURCE_GRAPH" gremlin-execute --script 'g.V().count()' +bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" gremlin-execute --script 'g.V().count()' +bin/hugegraph --url "$SOURCE_URL" --graph "$SOURCE_GRAPH" gremlin-execute --script 'g.E().count()' +bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" gremlin-execute --script 'g.E().count()' ``` -该命令用于恢复图模式为 NONE。 - -至此，一次完整的图备份和图恢复流程结束。 +两组计数结果应保持完全一致。建议进一步通过 [Schema API](/cn/docs/clients/restful-api/schema/) 核对元数据定义， +并抽样检查顶点和边的属性与端点关联，确保业务数据的完整性。 #### 帮助 -备份和恢复命令的详细使用方式可以参考[hugegraph-tools文档](/cn/docs/quickstart/toolchain/hugegraph-tools)。 +更多关于压缩、并发和分类型备份的参数说明，见 [Tools 参考](/cn/docs/quickstart/toolchain/hugegraph-tools)。 +在跨大版本恢复前，建议先在测试环境中验证备份文件的兼容性。 ## Backup/Restore使用和实现的API说明 #### Backup -Backup 使用`元数据`和`图数据`的相应的 list(GET) API 导出，并未增加新的 API。 +Backup 使用 Schema、顶点和边的列表 GET API 导出数据。 #### Restore -Restore 使用`元数据`和`图数据`的相应的 create(POST) API 导入，并未增加新的 API。 - -Restore 时存在两种不同的模式： Restoring 和 Merging，另外，还有常规模式 NONE(默认)，区别如下： - -- None 模式，元数据和图数据的写入属于正常状态，可参见功能说明。特别的： - - 元数据（schema）创建时不允许指定 ID - - 图数据（vertex）在 id strategy 为 Automatic 时，不允许指定 ID -- Restoring 模式，恢复到一个新图中，特别的： - - 元数据（schema）创建时允许指定 ID - - 图数据（vertex）在 id strategy 为 Automatic 时，允许指定 ID -- Merging 模式，合并到一个已存在元数据和图数据的图中，特别的： - - 元数据（schema）创建时不允许指定 ID - - 图数据（vertex）在 id strategy 为 Automatic 时，允许指定 ID - -正常情况下，图模式为 None，当需要 Restore 图时，需要根据需要临时修改图模式为 Restoring 模式或者 Merging 模式，并在完成 Restore 时，恢复图模式为 None。 - -实现的设置图模式的 RESTful API 如下： +Restore 使用 Schema、顶点和边的创建 API 导入数据。模式对应的 ID 行为见[前面的选择表](#restore)。 +`RESTORING` 允许指定 Schema ID；`MERGING` 不允许指定 Schema ID。两种模式均允许保留 `AUTOMATIC` 顶点 ID。 +普通 `NONE` 模式不接受这些恢复操作所需的 ID 设置。 ##### 查看某个图的模式 @@ -128,7 +148,7 @@ GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/mode } ``` -> 合法的图模式包括：NONE，RESTORING，MERGING，LOADING +合法图模式为 `NONE`、`RESTORING`、`MERGING` 和 `LOADING`。 ##### 设置某个图的模式 @@ -146,7 +166,7 @@ PUT http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/mode "RESTORING" ``` -> 合法的图模式包括：NONE，RESTORING，MERGING，LOADING +合法图模式为 `NONE`、`RESTORING`、`MERGING` 和 `LOADING`。 ###### Response Status diff --git a/content/cn/docs/guides/faq.md b/content/cn/docs/guides/faq.md index ae41d1404d..ceaacfe531 100644 --- a/content/cn/docs/guides/faq.md +++ b/content/cn/docs/guides/faq.md @@ -4,98 +4,139 @@ linkTitle: "FAQ" weight: 6 --- -- 如何选择后端存储? 选 RocksDB 还是分布式存储? +## 部署与启动 {#deployment} - HugeGraph 支持多种部署模式，根据数据规模和场景选择： - - **单机模式**：Server + RocksDB，适合开发测试、Graph AI 和中小规模生产环境（≤ 2 TB） - - **分布式模式**：HugeGraph-PD + HugeGraph-Store（HStore），用于需要水平扩展和多副本的部署，支持 ≤ 1 PB 的数据规模 +### 如何选择 RocksDB 或 HStore {#choose-backend} - 1.7.0 支持 RocksDB、HStore、HBase 和 Memory。Cassandra、ScyllaDB、MySQL、PostgreSQL 等旧后端需使用 1.5.x 或更早版本。 +开发测试或中小规模单机部署推荐选择 **Server + RocksDB**，部署轻便且性能优异。当数据规模较大（如 TB/PB 级）、需要水平扩容或多副本高可用时，推荐部署 **HStore 分布式集群**（PD + Store + Server）。 -- 启动服务时提示：`xxx (core dumped) xxx` - - 请先确认 JDK 版本符合对应版本要求：HugeGraph 1.8.0+ 要求 Java 17，1.7.0 要求 Java 11。HugeGraph 不再支持 Java 8。 +1.7.0 支持 RocksDB、HStore、HBase 和 Memory。Cassandra、ScyllaDB、MySQL、PostgreSQL 等旧后端需使用低于 1.7.0 的版本。 +部署步骤见 [Server 快速开始](/cn/docs/quickstart/hugegraph/hugegraph-server/)。 -- 启动服务成功了，但是操作图时有类似于"无法连接到后端或连接未打开"的提示 +### 启动时出现 core dumped 怎么查 {#core-dump} - RocksDB、HBase 等本地持久化后端首次启动前需要使用 `init-store` 初始化。HStore 由 PD、Store 管理，不执行该脚本。 +先执行 `java -version` 确认 JDK 版本：**HugeGraph master 分支（以 1.8 为基线）要求 Java 17**，**1.7.0 要求 Java 11**（已不再支持 Java 8）。 +若 JDK 版本正确，进一步查看 Server 日志及 JVM 崩溃日志（`hs_err_pid*.log`），排查是否由于系统架构不匹配、依赖的动态库（如 RocksDB JNI）缺失或系统内存不足引起。 -- 所有的后端在使用前都需要执行`init-store`吗，序列化的选择可以随意填写么? - - Memory 和 HStore 不执行 `init-store`；RocksDB、HBase 等本地持久化后端首次使用前需要初始化。序列化器必须与后端匹配，例如 RocksDB 使用 `binary`。 +### 哪些后端需要 init-store {#initialize-backend} -- 执行`init-store`报错：```Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp/librocksdbjni3226083071221514754.so: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.10' not found (required by /tmp/librocksdbjni3226083071221514754.so)``` +使用 RocksDB 或 HBase 等本地持久化后端时，**首次启动前必须执行 `bin/init-store.sh`** 进行初始化。Memory（内存后端）和 HStore（分布式后端）无需执行此脚本，HStore 的存储由 PD 和 Store 统一管理。 - RocksDB需要 gcc 4.3.0 (GLIBCXX_3.4.10) 及以上版本 +注意：`backend` 与 `serializer` 必须配套使用（例如 RocksDB 配套 `binary`），不可随意替换。若服务启动后提示无法连接后端，请优先检查是否完成初始化以及后端存储配置是否正确。 -- `bin`目录下包含`start-hugegraph.sh`、`start-restserver.sh`和`start-gremlinserver.sh`三个似乎与启动有关的脚本，到底该使用哪个 +### init-store 报 GLIBCXX 版本缺失怎么办 {#native-library} - 当前发布包只保留 `start-hugegraph.sh` 作为 Server 启动脚本。GremlinServer 和 REST Server 由同一进程启动。 +出现以下报错表示当前系统的 `libstdc++` 版本过低，缺少 RocksDB JNI 本地库所需的符号： -- 配置了两个图，名字是`hugegraph`和`hugegraph1`，而启动服务的命令是`start-hugegraph.sh`，是只打开了`hugegraph`这个图吗 +```text +java.lang.UnsatisfiedLinkError: .../librocksdbjni....so: +.../libstdc++.so.6: version `GLIBCXX_3.4.10' not found +``` - 脚本名称与图名无关。Server 默认从 `conf/graphs` 扫描并加载本地图配置，目录可在 `conf/rest-server.properties` 的 `graphs` 配置项中修改。`graph.load_from_local_config` 默认 - `false`，控制构造阶段预加载，不能用于禁止启动时加载。每个本地图使用独立配置文件，配置变化后重启 Server。详见[服务端配置说明](/cn/docs/config/config-guide/)。 +解决方法：核对发行包支持的系统架构，升级系统的 `libstdc++`（或安装较新的 GCC 运行库），或者直接使用官方提供的 Docker 镜像运行。 -- 服务启动成功后，使用`curl`查询所有顶点时返回乱码 +### 应该使用哪个 Server 启动脚本 {#start-script} - 服务端返回的批量顶点/边是压缩（gzip）过的，可以使用管道重定向至 `gunzip` 进行解压（`curl http://example | gunzip`），也可以用`Firefox`的`postman`或者`Chrome`浏览器的`restlet`插件发请求，会自动解压缩响应数据。 +当前发行包统一使用 `bin/start-hugegraph.sh`，GremlinServer 与 REST Server 会在同一个进程中启动。旧版本文档中的 `start-restserver.sh` 和 `start-gremlinserver.sh` 已废弃。 -- 使用顶点Id通过`RESTful API`查询顶点时返回空，但是顶点确实是存在的 +### 启动脚本会加载哪些图 {#load-graphs} - 检查顶点Id的类型，如果是字符串类型，`API`的`url`中的id部分需要加上双引号，数字类型则不用加。 +脚本名称与加载的图无关。Server 默认会扫描并加载 `conf/graphs` 目录下的所有图配置（可在 `conf/rest-server.properties` 中的 `graphs` 修改该目录）。每张本地图使用独立的配置文件，修改配置后需重启 Server。 -- 已经根据需要给顶点Id加上了双引号，但是通过`RESTful API`查询顶点时仍然返回空 - - 检查顶点id中是否包含`+`、`空格`、`/`、`?`、`%`、`&`和`=`这些URL的保留字符，如果存在则需要进行编码。下表给出了编码值： - - ```text - 特殊字符 | 编码值 - --------| ---- - + | %2B - 空格 | %20 - / | %2F - ? | %3F - % | %25 - # | %23 - & | %26 - = | %3D - ``` - -- 查询某一类别的顶点或边（`query by label`）时提示超时 +注：`graph.load_from_local_config` 默认为 `false`（控制构造阶段预加载），不能用来禁止启动时加载本地图。详见[服务端配置说明](/cn/docs/config/config-guide/)。 - 由于属于某一label的数据量可能比较多，请加上limit限制。 +## 连接与查询 {#queries} -- 通过`RESTful API`操作图是可以的，但是发送`Gremlin`语句就报错：`Request Failed(500)` +### curl 返回乱码怎么办 {#compressed-response} - 可能是`GremlinServer`的配置有误，检查`gremlin-server.yaml`的`host`、`port`是否与`rest-server.properties`的`gremlinserver.url`匹配，如不匹配则修改，然后重启服务。 +批量顶点和边接口默认会返回 gzip 压缩数据。使用 curl 请求时加上 `--compressed` 参数即可自动解压： -- 使用`Loader`导数据出现`Socket Timeout`异常，然后导致`Loader`中断 +```bash +curl --compressed \ + 'http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices?limit=10' +``` - 持续地导入数据会使`Server`的压力过大，然后导致有些请求超时。可以通过调整`Loader`的参数来适当缓解`Server`压力（如：重试次数，重试间隔，错误容忍数等），降低该问题出现频率。 +若开启了认证，需同时传入凭据。若返回内容仍不符合预期，请检查 HTTP 状态码、`Content-Type` 与 `Content-Encoding`，不要把 JSON 报错直接管道传给 `gunzip`。 -- 如何删除图中的全部数据 +### 按顶点 ID 查询却找不到顶点 {#vertex-id} - 管理员可调用 `DELETE /graphspaces/{graphspace}/graphs/{graph}/clear?confirm_message=I'm sure to delete all data`。`confirm_message` 查询参数必须与该值完全一致，否则请求会被拒绝，详见 [Graph API](../clients/restful-api/graphs)。该操作会清除 schema、顶点、边和索引。 +先从已有查询结果中确认 `id` 的实际格式和类型：**字符串 ID 需要 JSON 双引号，数值 ID 不需要**。例如字符串 ID `1:marko` 的 JSON 格式应为 `"1:marko"`。 -- 清空了数据库，并且执行了`init-store`，但是添加`schema`时提示"xxx has existed" +在把 ID 放入 URL 路径时，必须对包含双引号在内的完整字符串进行 URL 编码。若 ID 含有特殊字符，不能仅补双引号而遗漏编码： - `HugeGraphServer`内是有缓存的，清空数据库的同时是需要重启`Server`的，否则残留的缓存会产生不一致。 +| 字符 | URL 编码 | +| --- | --- | +| `+` | `%2B` | +| 空格 | `%20` | +| `/` | `%2F` | +| `?` | `%3F` | +| `%` | `%25` | +| `#` | `%23` | +| `&` | `%26` | +| `=` | `%3D` | +| `"` | `%22` | -- 插入顶点或边的过程中报错：`The max length of vertex id is 16384, but got xxx {yyy}` 或 `The max length of edge id is 65536, but got xxx {yyy}` +### 按类型查询顶点或边时超时 {#label-timeout} - 为了保证查询性能，目前的后端存储对id列的长度做了限制，顶点id不能超过16384字节，边id长度不能超过65536字节；索引id超过32字节时会转为哈希存储，而不是报错。 +当某个标签下的数据量很大时，全量扫描很容易耗尽资源导致超时。建议优化方式： +1. 加上 `limit()` 限制返回记录数； +2. 使用分页机制分批读取； +3. 配合具体属性过滤，并确保对应属性已建立索引，避免全表扫描。 -- 是否支持嵌套属性，如果不支持，是否有什么替代方案 +### REST 请求正常，Gremlin 返回 500 {#gremlin-connection} - 嵌套属性目前暂不支持。替代方案：可以把嵌套属性作为单独的顶点拿出来，然后用边连接起来。 +REST 接口正常仅说明 HTTP 入口可用，Gremlin 执行返回 500 通常有以下原因： +1. **Gremlin Server 连接异常**：检查 `conf/rest-server.properties` 中的 `gremlinserver.url` 是否指向可达的 Gremlin 端口（默认 `8182`，对应 `conf/gremlin-server.yaml`）； +2. **语句语法或逻辑错误**：例如未加索引导致遍历被拦截； +3. **认证凭据缺失或过期**。 -- 一个`EdgeLabel`是否可以连接多对`VertexLabel`，比如"投资"关系，可以是"个人"投资"企业"，也可以是"企业"投资"企业" +建议查看 `logs/hugegraph-server.log` 中的异常堆栈定位具体报错。 - 可以。创建`EdgeLabel`时对每一对顶点标签各调用一次`link(sourceLabel, targetLabel)`，所有配对都会被保留，因此同一个"投资"标签可以同时覆盖"个人"投资"企业"和"企业"投资"企业"。旧的`sourceLabel()`和`targetLabel()`构建方法已废弃，且只支持单一配对。 +### REST 请求返回 HTTP 415 {#content-type} -- 通过`RestAPI`发送请求时提示`HTTP 415 Unsupported Media Type` +发送带请求体的 POST/PUT 请求时，需在请求头中指定 `Content-Type: application/json`，并确保请求体是合法的 JSON 格式。 - 请求头中需要指定`Content-Type:application/json` +## 导入与数据维护 {#data-management} -其他问题可以在对应项目的 issue 区搜索，例如 [Server-Issues](https://github.com/apache/hugegraph/issues) / [Loader Issues](https://github.com/apache/hugegraph-toolchain/issues) +### Loader 出现 Socket Timeout {#loader-timeout} + +导入超时通常由于单批次写入过大或并发压力过高导致。建议： +1. 适当调小 `--batch-size`（如由 500 降为 200）； +2. 适当降低并发线程数； +3. 调大 `--timeout` 并配置重试机制。参数说明见 [Loader 指南](/cn/docs/quickstart/toolchain/hugegraph-loader/)。 + +### 如何清空一张图 {#clear-graph} + +管理员可调用以下接口清空指定图的 Schema、顶点、边和索引（执行前请务必确认目标图并做好备份）： + +```text +DELETE /graphspaces/{graphspace}/graphs/{graph}/clear +``` + +请求需携带 `confirm_message` 查询参数，值必须严格等于 `I'm sure to delete all data`（需进行 URL 编码）。完整调用方式见 [Graph API](/cn/docs/clients/restful-api/graphs/)。 + +### 清理底层存储后，为什么仍提示 Schema 已存在 {#schema-cache} + +Server 在内存中维护了 Schema 缓存。如果直接绕过 Server 清理底层存储（如清空 RocksDB 目录或 HBase 表），Server 缓存与实际存储将出现不一致。 +若已手动清理底层，必须**重启 Server**。日常清理建议使用官方提供的 [Clear 图 API](/cn/docs/clients/restful-api/graphs/#clear-graph)。 + +## 建模与 ID {#modeling} + +### 顶点或边 ID 超过长度限制 {#id-length} + +为保证底层存储与索引性能，HugeGraph 对 ID 长度作了如下限制（按编码后的字节数计算）： +- **顶点 ID**：最长 16384 字节（16 KB）； +- **边 ID**：最长 65536 字节（64 KB）。 + +建议使用简洁的业务唯一标识作为主键，避免在 ID 中拼接大段文本。注：属性索引值超过 32 字节时会自动转为哈希存储，不会报错超长。 + +### 是否支持嵌套属性 {#nested-properties} + +目前不支持嵌套属性（复杂对象或嵌套 Map）。如果需要表达多层级关系，建议将嵌套对象拆分成独立的顶点，通过边与主体顶点相连。 + +### 一个边类型能连接多组顶点类型吗 {#edge-links} + +支持。创建 `EdgeLabel` 时，对每一组起点和终点类型分别调用一次 `link(sourceLabel, targetLabel)` 即可。 +例如“投资”关系可以同时连接“个人 → 企业”和“企业 → 企业”。旧版 `sourceLabel()` 与 `targetLabel()` 只支持单一配对，现已废弃。 + +其他问题可搜索 [Server issues](https://github.com/apache/hugegraph/issues) 或 [Toolchain issues](https://github.com/apache/hugegraph-toolchain/issues)。 diff --git a/content/cn/docs/images/hubble/accounts.jpg b/content/cn/docs/images/hubble/accounts.jpg index df8d31583c..fb5cf939ac 100644 Binary files a/content/cn/docs/images/hubble/accounts.jpg and b/content/cn/docs/images/hubble/accounts.jpg differ diff --git a/content/cn/docs/images/hubble/algorithms.jpg b/content/cn/docs/images/hubble/algorithms.jpg index b243901d38..027307349c 100644 Binary files a/content/cn/docs/images/hubble/algorithms.jpg and b/content/cn/docs/images/hubble/algorithms.jpg differ diff --git a/content/cn/docs/images/hubble/cluster-overview.jpg b/content/cn/docs/images/hubble/cluster-overview.jpg index b99efc0819..70336b88d2 100644 Binary files a/content/cn/docs/images/hubble/cluster-overview.jpg and b/content/cn/docs/images/hubble/cluster-overview.jpg differ diff --git a/content/cn/docs/images/hubble/graphspace.jpg b/content/cn/docs/images/hubble/graphspace.jpg index 7b887affbb..45eb97ba37 100644 Binary files a/content/cn/docs/images/hubble/graphspace.jpg and b/content/cn/docs/images/hubble/graphspace.jpg differ diff --git a/content/cn/docs/images/hubble/members.jpg b/content/cn/docs/images/hubble/members.jpg index 31d86bd6cb..3402a71886 100644 Binary files a/content/cn/docs/images/hubble/members.jpg and b/content/cn/docs/images/hubble/members.jpg differ diff --git a/content/cn/docs/images/hubble/overview.jpg b/content/cn/docs/images/hubble/overview.jpg index de4db52d29..e9f0ee50f4 100644 Binary files a/content/cn/docs/images/hubble/overview.jpg and b/content/cn/docs/images/hubble/overview.jpg differ diff --git a/content/cn/docs/images/hubble/query.jpg b/content/cn/docs/images/hubble/query.jpg index becd373fcf..55dd1f433c 100644 Binary files a/content/cn/docs/images/hubble/query.jpg and b/content/cn/docs/images/hubble/query.jpg differ diff --git a/content/cn/docs/images/hubble/schema.jpg b/content/cn/docs/images/hubble/schema.jpg index a868f3508a..34e68c7231 100644 Binary files a/content/cn/docs/images/hubble/schema.jpg and b/content/cn/docs/images/hubble/schema.jpg differ diff --git a/content/cn/docs/images/hubble/store-node.jpg b/content/cn/docs/images/hubble/store-node.jpg index 523e37edea..387ea5280a 100644 Binary files a/content/cn/docs/images/hubble/store-node.jpg and b/content/cn/docs/images/hubble/store-node.jpg differ diff --git a/content/cn/docs/images/hubble/text2gql-prompt.jpg b/content/cn/docs/images/hubble/text2gql-prompt.jpg index 2a92dc199f..6ae93bd6bb 100644 Binary files a/content/cn/docs/images/hubble/text2gql-prompt.jpg and b/content/cn/docs/images/hubble/text2gql-prompt.jpg differ diff --git a/content/cn/docs/images/hubble/text2gql-settings.jpg b/content/cn/docs/images/hubble/text2gql-settings.jpg index a36dd2e55b..d15df46f49 100644 Binary files a/content/cn/docs/images/hubble/text2gql-settings.jpg and b/content/cn/docs/images/hubble/text2gql-settings.jpg differ diff --git a/content/cn/docs/images/hubble/text2gql.jpg b/content/cn/docs/images/hubble/text2gql.jpg index f9f020bc01..c154ec2214 100644 Binary files a/content/cn/docs/images/hubble/text2gql.jpg and b/content/cn/docs/images/hubble/text2gql.jpg differ diff --git a/content/cn/docs/quickstart/client/hugegraph-client.md b/content/cn/docs/quickstart/client/hugegraph-client.md index 0f5019821c..a832a0e927 100644 --- a/content/cn/docs/quickstart/client/hugegraph-client.md +++ b/content/cn/docs/quickstart/client/hugegraph-client.md @@ -12,31 +12,32 @@ weight: 1 ### 2 环境要求 -- JDK 11（当前 CI 使用版本；源码目标版本为 Java 8） -- Maven 3.6+ +- JDK 17，用于本文示例及从 Toolchain `master` 构建 Client `1.8.0` +- Maven 3.6.3+ ### 3 使用流程 -使用 HugeGraph-Client 的基本步骤如下： +按下面的顺序接入 HugeGraph-Client。 -- 新建Eclipse/ IDEA Maven 项目； -- 在 pom 文件中添加 HugeGraph-Client 依赖； -- 创建类，调用 HugeGraph-Client 接口； +1. 使用 IntelliJ IDEA 或 Eclipse 新建 Maven 工程。 +1. 在 `pom.xml` 中添加 HugeGraph-Client 依赖。 +1. 创建客户端、调用 API，并在使用后关闭客户端。 +{.steps} -详细使用过程见下节完整示例。 +详细操作见下面的完整示例。 ### 4 完整示例 #### 4.1 新建 Maven 工程 -可以选择 Eclipse 或者 Intellij Idea 创建工程： +使用 Eclipse 或 IntelliJ IDEA 创建 Maven 工程。 - [Eclipse 新建 Maven 工程](http://www.vogella.com/tutorials/EclipseMaven/article.html) - [Intellij Idea 创建 maven 工程](https://vaadin.com/docs/-/part/framework/getting-started/getting-started-idea.html) #### 4.2 添加 hugegraph-client 依赖 -添加 hugegraph-client 依赖 +在 `pom.xml` 中添加依赖。 ```xml @@ -49,24 +50,41 @@ weight: 1 ``` -以下完整示例使用的 Schema、Graph、Gremlin 和资源关闭接口在已发布的 Client `1.7.0` 中可用，也与 Toolchain `master` 兼容。 -上面的依赖可直接用于示例工程。 -如果需要[API 参考](/cn/docs/clients/hugegraph-client/)中的 master 新接口（例如能力探测），请使用 Toolchain `master` 的 Client `1.8.0`， -将 POM 中的版本改为 `1.8.0`。该版本未发布到 Maven 仓库时，先在 Toolchain 源码根目录运行以下命令， -将 Client 及其父 POM 安装到本地 Maven 仓库。源码构建仍需从配置的 Maven 仓库获取依赖。 +先根据用途选择客户端版本。 + +| 用途 | 客户端版本 | Java 编译目标 | +| --- | --- | --- | +| 运行下方完整示例 | 已发布的 `1.7.0`，使用上面的依赖即可 | 发布版 POM 为 Java 8；本文统一使用 JDK 17 | +| 使用能力探测等 master 新接口 | Toolchain `master`（以 1.8 为基线） | Java 17 | + +示例使用的 Schema、Graph、Gremlin 和资源关闭接口在 Client `1.7.0` 中可用，也与 Toolchain `master` 兼容。 +发布版的 Java 编译目标不代表所有传递依赖的运行环境要求。 + +若需要使用 master 分支未发布的新接口，可从源码编译安装 Client（需配合 Toolchain 源码，详见 [Tools 源码构建指引](/cn/docs/quickstart/toolchain/hugegraph-tools/#build-current-source)）： ```bash -mvn install -pl hugegraph-client -am -DskipTests -Dmaven.javadoc.skip=true -ntp +java -version +mvn -version +mvn -Dmaven.repo.local="${candidate_dir:?Run the candidate SDK setup first}/m2" \ + -Dsdk.validation.mode=candidate \ + install -pl hugegraph-client -am -DskipTests -Dmaven.javadoc.skip=true -ntp ``` -Client 与 Server 的版本不必相同；升级前核对服务端 API 兼容范围及所需功能。 +两条版本检查都应显示 JDK 17。使用 master 分支 Client 的项目运行 Maven 命令时，也需添加 `-Dmaven.repo.local="$candidate_dir/m2"`。 +Client 与 Server 的版本不必相同；升级前核对所需 API 及兼容范围。 #### 4.3 Example +选择单条操作或批量操作示例，将完整代码保存为 Maven 工程中的 `src/main/java/SingleExample.java` 或 +`src/main/java/BatchExample.java`。复制按钮会包含已折叠的代码行。 + +{{< tabs group="java-client-example" default="single" label="Java 客户端示例" >}} +{{< tab label="SingleExample" value="single" >}} + ##### 4.3.1 SingleExample -```java +```java {collapse=30} import java.io.IOException; import java.util.Iterator; import java.util.List; @@ -218,9 +236,12 @@ public class SingleExample { } ``` +{{< /tab >}} +{{< tab label="BatchExample" value="batch" >}} + ##### 4.3.2 BatchExample -```java +```java {collapse=30} import java.util.ArrayList; import java.util.List; @@ -362,9 +383,17 @@ public class BatchExample { } ``` +{{< /tab >}} +{{< /tabs >}} + #### 4.4 运行 Example -运行 Example 之前需要启动 Server, 启动过程见[HugeGraph-Server 快速上手](/cn/docs/quickstart/hugegraph/hugegraph-server) +1. [启动 HugeGraph Server](/cn/docs/quickstart/hugegraph/hugegraph-server)，在图空间 `DEFAULT` 中准备名为 `hugegraph` 的测试图。 +1. 将 Maven 工程的 JDK 设为 17，把所选示例中 `HugeClient.builder(...)` 和 `configUser(...)` 的 Server 地址、图空间、图名和凭据改为实际配置。 + Server 未开启鉴权时，省略 `configUser(...)`。 +1. 在 IDE 中运行 `SingleExample.main()` 或 `BatchExample.main()`。两个示例都会创建 Schema 和图数据、打印结果并关闭客户端。 + 请使用测试图，已有 Schema 或数据可能影响重复运行。 +{.steps} #### 4.5 详细 API 说明 diff --git a/content/cn/docs/quickstart/hugegraph-ai/quick_start.md b/content/cn/docs/quickstart/hugegraph-ai/quick_start.md index 87f853337e..592bfc89f9 100644 --- a/content/cn/docs/quickstart/hugegraph-ai/quick_start.md +++ b/content/cn/docs/quickstart/hugegraph-ai/quick_start.md @@ -4,15 +4,55 @@ linkTitle: "LLM 使用流程" weight: 3 --- -本文说明 HugeGraph-LLM Web 页面的处理流程。服务启动方式见 [HugeGraph-LLM](./hugegraph-llm.md)。 +本页按 Web 页面的操作顺序，说明如何建立索引、查询文档和生成 Gremlin。服务启动方式见 [HugeGraph-LLM](./hugegraph-llm.md)。 + +先准备可连接的 HugeGraph Server、生成模型和 Embedding 模型。查看配置或在 `Graph Tools` 手写 Gremlin 不需要模型； +文档向量化需要 Embedding，图数据抽取和答案生成还需要 LLM。Reranker 可选，初次运行可用本地 `bleu`。 + +| 想做什么 | 页面入口 | 需要准备什么 | +| --- | --- | --- | +| 从文档建立索引 | `1. Build RAG Index` | 文档、Embedding；抽取图数据还需 Schema 和 LLM | +| 根据文档或图回答问题 | `2. (Graph)RAG & User Functions` | 对应索引和 LLM | +| 把问题转成 Gremlin | `3. Text2gremlin` | 图 Schema、LLM；引用示例还需 Embedding 和示例索引 | +| 直接验证图数据 | `4. Graph Tools` | HugeGraph 连接和手写 Gremlin | + +本文截图基于 HugeGraph-LLM Web 界面，示例包含中英文提示词模式（通过环境变量 `LANGUAGE` 切换内置提示词语言，界面控件保持英文）。 + +> [!DETAILS]- 源码启动时缺少批量问题模板 +> +> 若启动时报 `FileNotFoundError`，且缺少的文件为 `questions_template.xlsx`，在已安装依赖的仓库根目录创建输入模板后重启。 +> 已有文件会保留。模板包含完整的 6 列，只填写测试问题和预期答案，4 个模型回答列留空。 +> +> ```bash +> uv run --extra llm python - <<'PYTHON' +> from pathlib import Path +> from openpyxl import Workbook +> +> path = Path("hugegraph-llm/src/hugegraph_llm/resources/demo/questions_template.xlsx") +> if not path.exists(): +> workbook = Workbook() +> sheet = workbook.active +> sheet.append([ +> "Question", "Expected Answer", "Basic LLM Answer", +> "Vector-only Answer", "Graph-only Answer", "Graph-Vector Answer", +> ]) +> sheet.append(["Who is Sarah?", "Sarah is a 30-year-old attorney.", None, None, None, None]) +> workbook.save(path) +> PYTHON +> ``` ## 0. 配置面板 -标签页上方是可折叠的配置面板，共五个部分：`1. Set up the HugeGraph server.`、`2. Set up the LLM.`、`3. Set up the Embedding.`、`4. Set up the Reranker.` 和 `5. Set up the vector engine.`。每部分都有独立的应用按钮，应用后会把受支持的字段写回 `.env`。页面顶部还会显示当前提示词语言。 +标签页上方是可折叠的配置面板，共五个部分，包括 `1. Set up the HugeGraph server.`、`2. Set up the LLM.`、`3. Set up the Embedding.`、`4. Set up the Reranker.` 和 `5. Set up the vector +engine.`。每部分都有独立的应用按钮，应用后会把受支持的字段写回 `.env`。页面顶部还会显示当前提示词语言。 + +在 LLM 配置的 `chat`、`mini_tasks`、`text2gql` 子页分别设置回答、抽取和查询生成模型。 +截图使用本地 Ollama 的 `qwen2.5:3b` 和 `mxbai-embed-large`，向量引擎为 `Faiss`。 +更换 Embedding 模型后需重建索引，避免新旧向量维数或含义不同。 ## 1. 构建 RAG 索引 -第一个标签页负责两类索引： +第一个标签页负责两类索引。 - 将文档切分后写入 chunk 向量索引。 - 按 Schema 从文档抽取顶点和边，写入 HugeGraph，并维护顶点向量索引。 @@ -29,38 +69,100 @@ flowchart TD 输入来自 `text` 子页或 `file` 子页。上传支持 `.txt`、`.docx` 和 `.pdf`，可一次选择多个文件。 -页面包含文档、Schema、抽取提示词和结果区域。常用操作有： +首次尝试可以使用下面的短文，填入 `Input Doc(s)`。请先选择专用测试图，避免将示例写入业务图。 + +```text +Sarah is a 30-year-old attorney. James is a journalist. Sarah has been James's roommate since 2010. +``` + +将下面的 JSON 填入 `Graph Schema`，保留默认抽取提示词。它只允许抽取人物及室友关系，年龄和职业可为空。 + +```json +{ + "vertexlabels": [ + {"id": 1, "name": "person", "id_strategy": "PRIMARY_KEY", + "primary_keys": ["name"], "properties": ["name", "age", "occupation"], + "nullable_keys": ["age", "occupation"]} + ], + "edgelabels": [ + {"name": "roommate", "source_label": "person", "target_label": "person", "properties": ["date"]} + ], + "propertykeys": [ + {"name": "name", "data_type": "TEXT", "cardinality": "SINGLE"}, + {"name": "age", "data_type": "INT", "cardinality": "SINGLE"}, + {"name": "occupation", "data_type": "TEXT", "cardinality": "SINGLE"}, + {"name": "date", "data_type": "TEXT", "cardinality": "SINGLE"} + ] +} +``` + +页面包含文档、Schema、抽取提示词和结果区域。按下面的顺序建立图索引。只用向量检索时，完成第 1 步即可。 + +1. 点击 `Import into Vector`，切分文档并建立 chunk 向量索引。 +2. 在 `Graph Schema` 填入 Schema JSON 或已有图名，再点击 `Extract Graph Data (1)`。检查 `Output Info` 中的顶点、边和属性。 +3. 确认抽取结果后点击 `Load into GraphDB (2)`，写入 HugeGraph 并更新顶点向量。 +4. 打开 `Graph Tools`，执行 `g.V().count()` 和 `g.E().count()`，再查具体属性，确认数据已写入。 +{.steps} -1. `Import into Vector`：切分文档并建立 chunk 向量索引。 -2. `Extract Graph Data (1)`：按 Schema 抽取图数据。 -3. `Load into GraphDB (2)`：把抽取结果写入 HugeGraph，并自动更新顶点向量。 -4. `Update Vid Embedding`：重新生成顶点向量，通常只在图中已有数据时才需要单独执行。 +本例应得到 2 个顶点和 1 条边，Sarah 的年龄为 `30`，职业为 `attorney`。 +图里已经有数据时，可以单独点击 `Update Vid Embedding` 更新顶点索引。 这些按钮旁的 `Graph Extraction Split Type` 下拉框可选 `document`、`paragraph` 或 `sentence`。`document` 把输入整体作为一个单元，另外两种会在抽取前先切分长文档。 页面还可以查看或清除 chunk 索引、顶点索引和图数据。清除操作会删除已有数据，执行前先确认当前图和索引是否仍被其他查询使用。 -主控件下方还有两个折叠的辅助工具： +主控件下方还有两个折叠的辅助工具。 -- `Graph Schema Generator`：根据查询示例和少样本示例生成 Schema，填入 Graph Schema 字段。 -- `Graph Extraction Prompt Generator`：根据期望场景（例如社交关系、金融知识图谱）和选定的参考示例生成 Graph Extract Prompt Header。 +- `Graph Schema Generator` 根据查询示例和少样本示例生成 Schema，填入 Graph Schema 字段。 +- `Graph Extraction Prompt Generator` 根据期望场景（例如社交关系、金融知识图谱）和选定的参考示例生成 Graph Extract Prompt Header。 ## 2. GraphRAG 查询 -第二个标签页提供四种回答范围： +第二个标签页提供四种回答范围，按需要选择对应索引。 -- 直接使用 LLM 回答。 -- 只使用 chunk 向量召回。 -- 只使用图召回。 -- 合并图召回与向量召回。 +| 控件 | 回答依据 | +| --- | --- | +| `Basic LLM Answer` | 直接调用 LLM，不查询文档或图 | +| `Vector-only Answer` | chunk 向量检索结果 | +| `Graph-only Answer` | 图检索结果 | +| `Graph-Vector Answer` | 图与 chunk 向量检索结果 | + +推荐优先使用 `Graph-Vector Answer`，同时结合图数据检索与文档向量索引以获得更完整的事实支撑。其他范围的控件含义见上表。 + +截图使用下面的简短提示词。分别填入 `Query Prompt` 和 `Keywords Extraction Prompt`，其中占位符会由服务替换。 +关键词输出需使用代码块中的英文前缀和冒号，姓名的大小写应与图数据一致。 + +```text +只根据检索到的上下文，用中文简洁回答问题，保留原文中的姓名。上下文不足时说明缺少信息。 +上下文: {context_str} +问题: {query_str} +``` + +```text +从问题中提取人名，保留姓名原有的大小写，最多输出 {max_keywords} 个。 +只返回一行，严格使用英文冒号，格式为 KEYWORDS:姓名:1.0 +问题: {question} +``` + +1. 输入 `Who is Sarah?`，将 `Graph-only Answer` 设为 `False`，`Graph-Vector Answer` 设为 `True`。 +2. 初次验证时保留 `Template Num=-1` 和 `Rerank method=bleu`，点击 `Answer Question`。 +3. 查看 `Graph-Vector Answer`，将答案中的职业、年龄等事实与原文和图数据核对。 +{.steps} + +```gallery +![GraphRAG 回答范围与图向量联合检索答案](/images/docs/hugegraph-ai/cn/graphrag.jpg) +![GraphRAG 的中文提示词](/images/docs/hugegraph-ai/cn/query-prompts.jpg) +``` ```mermaid flowchart TD Q[问题] --> V[查询 chunk 向量索引] Q --> K[抽取关键词] K --> M[匹配图顶点] - M --> T[生成并执行 Gremlin] - T -->|失败| B[BFS 图遍历回退] + M --> D{启用 Text2Gremlin} + D -->|是| T[生成并执行 Gremlin] + D -->|否| B[BFS 图遍历] + T -->|失败| B T --> R[整理图结果] B --> R V --> S[合并与重排序] @@ -70,11 +172,11 @@ flowchart TD 图召回先用关键词精确匹配 HugeGraph 顶点，找不到时再用顶点向量做近似匹配。匹配结果会进入 Text2Gremlin；生成或执行失败时，流程可以回退到预定义的图遍历。 -`Template Num` 控制 Text2Gremlin 在图召回中的参与方式： +`Template Num` 控制 Text2Gremlin 在图召回中的参与方式。 -- 小于 0：完全跳过 Text2Gremlin，图召回直接使用预定义的图遍历。 -- 等于 0：不带任何示例生成 Gremlin（zero-shot）。 -- 大于 0：从示例索引中取相应数量的相近示例，并采用带模板的生成结果。示例数量会被限制在 0 到 10 之间。 +- 小于 0 时，完全跳过 Text2Gremlin，图召回直接使用预定义的图遍历。 +- 等于 0 时，不检索参考示例，使用基础生成结果。 +- 大于 0 时，从示例索引中取相应数量的相近示例，并采用带模板的生成结果。示例数量会被限制在 0 到 10 之间。 该标签页的其他控件还有 `Rerank method`（`bleu` 或 `reranker`）、`Graph Ratio`、`Near neighbor first` 和 `Query related information`，以及可编辑的 `Query Prompt` 和 `Keywords Extraction Prompt`。 @@ -82,24 +184,55 @@ flowchart TD ## 3. Text2Gremlin -第三个标签页分为两部分。上半部分用问题与 Gremlin 对照文件（`.json` 或 `.csv`）构建示例向量索引；未上传文件时使用内置的 `resources/demo/text2gremlin.csv`。 +第三个标签页将自然语言转换为 Gremlin，并执行生成的查询。上半部分用 `.json` 或 `.csv` 的问题与 Gremlin 对建立示例向量索引； +未上传文件时使用内置的 `resources/demo/text2gremlin.csv`。示例中的标签和属性应与当前图一致。 + +> [!TIP] +> 若上传 CSV 构建示例向量索引后未自动召回示例模板，可直接在下面的 Prompt 模板中手动提供 Few-shot 参考示例。 + +复现 Sarah 查询时，使用以下输入。 + +1. 将下面的 CSV 保存为 `sarah-gremlin.csv`，上传后点击 `Build Example Vector Index`，并把 `Number of refer examples` 设为 `1`。 +2. 在 `Schema` 填入下方的简化 Schema，只保留查询所需的类型与属性。 +3. 在 `Prompt` 填入下方的提示词，输入问题 `Who is Sarah?`，再点击 `Text2Gremlin`。 +4. 检查生成语句及两组 `Query ... Output`。本例返回 Sarah 的 `age=30` 和 `occupation=attorney`。 +{.steps} -下半部分把自然语言转换成 Gremlin： +```csv +query,gremlin +Who is Sarah?,"g.V().has('person', 'name', 'Sarah').valueMap()" +``` + +```json +{"vertexlabels":[{"name":"person","properties":["name","age","occupation"]}],"edgelabels":[]} +``` -1. 读取当前图的 Schema。 -2. 从示例向量索引取回相近的自然语言与 Gremlin 对。 -3. 把问题、Schema、示例和已匹配顶点填入提示词。 -4. 调用 LLM 生成 Gremlin，并按所选输出类型决定是否执行。 +```text +把问题转换为 HugeGraph 的只读 Gremlin 查询。 +按人物姓名查询时，使用 g.V().has('person', 'name', '<问题中的姓名>').valueMap()。 +保留姓名的大小写，按姓名查询，不使用顶点 ID。 +问题示例: Who is Peter? +Gremlin 示例: g.V().has('person', 'name', 'Peter').valueMap() +图 Schema: {schema} +其他示例: {example} +已知顶点: {vertices} +问题: {query} +只返回一个 ```gremlin``` 代码块。 +``` -`Number of refer examples` 设置取回的示例数量，范围 0 到 10，默认 2。结果显示在四个字段中：带模板的 Gremlin、不带模板的 Gremlin，以及两者各自的执行输出。 +```gallery +![Text2Gremlin 生成的查询与执行结果](/images/docs/hugegraph-ai/cn/text2gremlin.jpg) +``` -![RAG 查询范围选择](/images/docs/hugegraph-ai/quick-start-03.jpg) +小模型可能生成错误 ID、大小写不符的姓名或不满足索引条件的过滤语句。应查看执行结果中的具体数据，再判断查询是否成功。 +Web 页面会直接执行生成的语句，请使用专用测试图。需要只生成语句时，在 [REST API](./rest-api.md) 中选择相应输出类型。 -自定义提示词必须包含 `{query}`、`{schema}`、`{example}` 和 `{vertices}`。缺少任一占位符时，REST API 会拒绝请求。 +自定义提示词必须保留 `{query}`、`{schema}`、`{example}` 和 `{vertices}`。缺少任一占位符时，REST API 会拒绝请求。 ## 4. 图工具与管理工具 -`Graph Tools` 标签页可直接对当前图执行 Gremlin 查询、手动触发图备份，并通过 beta 操作初始化 HugeGraph 演示数据。后台还有两个任务：每天 01:00 自动备份图数据，以及在进程运行期间持续更新顶点 id 向量。 +`Graph Tools` 标签页可直接对当前图执行 Gremlin 查询、手动触发图备份，并通过 beta 操作初始化 HugeGraph 演示数据。后台还有两个任务，每天凌晨 1 点自动备份图数据，以及在进程运行期间持续更新顶点 id 向量。 +初始化演示数据会先清空当前图，只能在专用测试图上使用。 `Admin Tools` 需要密码。输入已配置的 `ADMIN_TOKEN` 后可查看 `logs/llm-server.log` 的末尾内容（每 60 秒自动刷新），并可手动刷新或清空该文件。`ADMIN_TOKEN` 为空或仍是占位值 `xxxx` 时，访问会被拒绝。 @@ -108,11 +241,13 @@ flowchart TD > [!WARNING] > 生产环境必须启用 HugeGraph-LLM 自身的登录认证，并替换 `USER_TOKEN` 和 `ADMIN_TOKEN`，同时在防火墙或网络入口设置来源 IP 白名单。HugeGraph Server 还必须单独开启认证与授权（见[认证与授权说明](/cn/docs/config/config-authentication/)）并保留 Server 审计日志（标准日志文件为 `audit-*.log`），为 `GRAPH_USER` 配置最小必要权限。AI 服务 token 不会替代 HugeGraph Server 认证。 -![RAG 界面中抽取的关键词](/images/docs/hugegraph-ai/quick-start-04.png) +```gallery +![Graph Tools 中手写 Gremlin 的查询结果](/images/docs/hugegraph-ai/cn/graph-tools.jpg) +``` ## 5. 提示词语言 -在 `hugegraph-llm/.env` 中设置： +在 `hugegraph-llm/.env` 中选择一种提示词语言。 ```properties # 英文提示词 @@ -122,7 +257,7 @@ LANGUAGE=EN LANGUAGE=CN ``` -修改后重启服务。该配置选择内置提示词语言，不会自动翻译输入文档，也不是 `/rag` 请求体字段。 +修改后重启服务。该配置选择内置提示词语言，不会自动翻译输入文档，也不属于 `/rag` 请求体字段。 ## 6. REST 调用 diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-pd.md b/content/cn/docs/quickstart/hugegraph/hugegraph-pd.md index 337a635881..669b236835 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-pd.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-pd.md @@ -121,7 +121,7 @@ PD 启动脚本读取安装目录中的 `conf/application.yml`。请使用与安 #### 4.1 配置项参考 -以下表格核对的是 server 主线提交 `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4` 中的 `hg-pd-dist` 配置和 Java 默认注入值（三个 `raft.rpc-*` 配置项按 [apache/hugegraph#3368](https://github.com/apache/hugegraph/pull/3368) 核对），不是 1.7.0 发布标签的全部行为。1.7.0 二进制包请使用其标签对应的原始配置；主线源码构建包则以 [主线配置文件](https://github.com/apache/hugegraph/blob/master/hugegraph-pd/hg-pd-dist/src/assembly/static/conf/application.yml) 为准。 +以下表格列出主线配置及代码默认值，Raft 超时设置可参阅 [PDConfig 实现](https://github.com/apache/hugegraph/blob/master/hugegraph-pd/hg-pd-core/src/main/java/org/apache/hugegraph/pd/config/PDConfig.java)。1.7.0 二进制包请使用对应标签的原始配置；主线源码构建包则以 [主线配置文件](https://github.com/apache/hugegraph/blob/master/hugegraph-pd/hg-pd-dist/src/assembly/static/conf/application.yml) 为准。下文会单独说明与 1.7.0 的差异。 **gRPC 与 REST** diff --git a/content/cn/docs/quickstart/hugegraph/hugegraph-server.md b/content/cn/docs/quickstart/hugegraph/hugegraph-server.md index 9ef4904a00..70a7a94abe 100644 --- a/content/cn/docs/quickstart/hugegraph/hugegraph-server.md +++ b/content/cn/docs/quickstart/hugegraph/hugegraph-server.md @@ -79,7 +79,8 @@ Server 安全检查（`-s true`）要求 Java 17 到 23 以及可读的 `conf/ja serializer: className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1 config: - ioRegistries: [org.apache.hugegraph.structure.io.HugeGraphIoRegistry] + builder: org.apache.hugegraph.io.HugeGraphTypeSerializerRegistryBuilder + ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry] ``` ```yaml {tab="GraphSON (HTTP/兼容)"} diff --git a/content/cn/docs/quickstart/toolchain/hugegraph-hubble.md b/content/cn/docs/quickstart/toolchain/hugegraph-hubble.md index 7a9abfeee8..6281323ffc 100644 --- a/content/cn/docs/quickstart/toolchain/hugegraph-hubble.md +++ b/content/cn/docs/quickstart/toolchain/hugegraph-hubble.md @@ -11,47 +11,69 @@ Hubble 是 HugeGraph 的 Web 管理与图可视化界面。你可以在同一个 并在图、表格和 JSON 视图之间自由切换。本文以 **单机 RocksDB Server + Hubble** 为例走通完整操作，无需预先部署 PD 或 Store。 如需搭配 HStore 分布式集群使用，可先了解本文的基础操作，再参考 [HStore 分布式补充](/cn/docs/quickstart/toolchain/visualization/hugegraph-hubble-hstore/)。 -文档内容以 Toolchain 与 Server 的 `master` 分支（当前版本均为 `1.8.0`）为准；Docker `latest` 为动态标签，实际部署建议使用固定版本。 +文档内容以 Toolchain 与 Server 的 `master` 分支（当前均以 1.8 为基线，尚未正式发布）为准；Docker `latest` 为动态标签，生产环境建议使用已发布的固定版本。 + +本文截图基于 HugeGraph Hubble 界面，中英文界面功能与操作一致。 > [!WARNING] > 请勿将未配置防护的 Hubble 或 Server 直接暴露在公网上。生产环境请务必开启 HTTPS、容器网络隔离、 > [认证与授权](/cn/docs/config/config-authentication/)及访问白名单。 +可按需快速跳转到对应操作；首次使用建议从启动单机环境开始： + +- [启动并加载示例](#启动单机组合) + + 启动两个服务，加载人物与软件示例图。 +- [查询与可视化](#gql-工作台查询与探索人物关系) + + 运行 Gremlin，再查看图、表格或 JSON 结果。 +- [导入自己的数据](#数据源管理把原始数据准备好) + + 准备 CSV，配置数据源、字段映射并验证导入结果。 +{.cards} + ## 启动单机组合 直接使用主仓库提供的 [docker/docker-compose.yml](https://github.com/apache/hugegraph/blob/master/docker/docker-compose.yml) 即可， 无需自行编写 Compose 配置文件。该文件已内置编排了 RocksDB Server 与 Hubble，并配置好网络、健康检查及持久化数据卷； 完整部署说明见同目录的 [README](https://github.com/apache/hugegraph/blob/master/docker/README.md)。 -```bash -git clone --branch master --single-branch --depth 1 https://github.com/apache/hugegraph.git -cd hugegraph/docker -``` +1. **准备官方 Compose 配置。** -若本地已有主仓库，直接进入其 `docker/` 目录即可。Compose 会自动挂载该目录下的 -[`conf/hubble/standalone.properties`](https://github.com/apache/hugegraph/blob/master/docker/conf/hubble/standalone.properties)， -已预设 `pd.enabled=false` 与 `server.direct_url=http://server:8080`。两项服务处于同一 Docker 网络中通信， -无需为每个图单独配置 Server 地址。请直接在 `docker/` 目录下启动，以确保相对配置路径有效。 + ```bash + git clone --branch master --single-branch --depth 1 https://github.com/apache/hugegraph.git + cd hugegraph/docker + ``` -Hubble 默认只在宿主机回环地址开放 `8088`（由 `HUBBLE_PUBLISH_HOST` 环境变量控制，默认为 `127.0.0.1`）； -Server 默认对外发布 `8080`。若仅供本机试用，可将 Compose 中 Server 的 `ports` 改为 `127.0.0.1:8080:8080`，避免匿名端口暴露到外部网络。 + 若本地已有主仓库，直接进入其 `docker/` 目录即可。Compose 会自动挂载该目录下的 + [`conf/hubble/standalone.properties`](https://github.com/apache/hugegraph/blob/master/docker/conf/hubble/standalone.properties)， + 已预设 `pd.enabled=false` 与 `server.direct_url=http://server:8080`。两项服务处于同一 Docker 网络中通信， + 无需为每个图单独配置 Server 地址。请直接在 `docker/` 目录下启动，以确保相对配置路径有效。 -为本次试用指定一个独立的项目名，后续命令在同一终端中沿用这些环境变量；另起终端时请重新设置： + Hubble 默认只在宿主机回环地址开放 `8088`（由 `HUBBLE_PUBLISH_HOST` 环境变量控制，默认为 `127.0.0.1`）； + Server 默认对外发布 `8080`。若仅供本机试用，可将 Compose 中 Server 的 `ports` 改为 `127.0.0.1:8080:8080`，避免匿名端口暴露到外部网络。 -```bash -export HUGEGRAPH_VERSION=latest -export HUBBLE_IMAGE=hugegraph/hubble:latest -export HUBBLE_DEMO_PROJECT="hubble-demo-$(date +%Y%m%d-%H%M%S)" -docker compose ls -docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml pull -docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml up -d --wait -docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml ps -curl -fsS http://127.0.0.1:8080/versions -``` +1. **启动并检查服务。** + + 为本次试用指定一个独立的项目名，后续命令在同一终端中沿用这些环境变量；另起终端时请重新设置。 -服务启动完成后，在浏览器访问 。若未在 `.env` 中配置 `HUGEGRAPH_ADMIN_PASSWORD`，Server 默认允许匿名访问， -Hubble 会直接进入首页。若已启用认证，请根据 Docker README 在 `.env` 中配置管理员密码与 JWT 密钥，并使用 Server 账号登录（Hubble 无独立用户库）。 -个人中心与账号管理入口仅在开启认证且具备相应权限时展示。 + ```bash + export HUGEGRAPH_VERSION=latest + export HUBBLE_IMAGE=hugegraph/hubble:latest + export HUBBLE_DEMO_PROJECT="hubble-demo-$(date +%Y%m%d-%H%M%S)" + docker compose ls + docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml pull + docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml up -d --wait + docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml ps + curl -fsS http://127.0.0.1:8080/versions + ``` + +1. **打开 Hubble。** + + 服务启动完成后，在浏览器访问 。若未在 `.env` 中配置 `HUGEGRAPH_ADMIN_PASSWORD`，Server 默认允许匿名访问， + Hubble 会直接进入首页。若已启用认证，请根据 Docker README 在 `.env` 中配置管理员密码与 JWT 密钥，并使用 Server 账号登录（Hubble 无独立用户库）。 + 个人中心与账号管理入口仅在开启认证且具备相应权限时展示。 +{.steps} `latest` 镜像便于快速体验最新功能，生产环境建议指定发布版本号或镜像 digest。正式发布包可前往 [下载页](/cn/docs/download/download/) 获取。 @@ -62,7 +84,7 @@ Hubble 会直接进入首页。若已启用认证，请根据 Docker README 在 Compose 中的 `server-data` 与 `hubble-data` 卷分别持久化保存图数据与 Hubble 元数据；更多生产配置参见 [Server 部署指南](/cn/docs/quickstart/hugegraph/hugegraph-server/)。 -## 首页：找到当前任务的入口 +## 首页与任务入口 {#首页找到当前任务的入口} 首页按“图概览 → 图导入 → 图查询”组织核心工作流。左侧导航栏可随时切换各功能模块； 顶部的图选择器用于切换当前操作的目标图，所有查询、Schema 操作和异步任务均作用于当前选中的图。 @@ -85,12 +107,12 @@ Compose 中的 `server-data` 与 `hubble-data` 卷分别持久化保存图数据 图概览、Schema 和查询工作区如下，点击任一截图可查看原图。 ```gallery -![人物与软件示例的图概览](/cn/docs/images/hubble/overview.jpg) # 图概览：选择图并查看示例数据 -![顶点类型列表中的主键与数值 ID 策略](/cn/docs/images/hubble/schema.jpg) # Schema：查看顶点类型、主键与 ID 策略 -![Gremlin 路径查询与图结果](/cn/docs/images/hubble/query.jpg) # 图查询：执行 Gremlin 并探索图结果 +![人物与软件示例的图概览](/cn/docs/images/hubble/overview.jpg) # 图概览，选择图并查看示例数据 +![顶点类型列表中的主键与数值 ID 策略](/cn/docs/images/hubble/schema.jpg) # Schema，查看顶点类型、主键与 ID 策略 +![Gremlin 路径查询与图结果](/cn/docs/images/hubble/query.jpg) # 图查询，执行 Gremlin 并探索图结果 ``` -## 图概览与图详情：先认识一张图 +## 认识图概览与图详情 {#图概览与图详情先认识一张图} 在【图概览】中选择默认图 `hugegraph`。点击图卡片或详情，可查看该图的 Schema 结构与数据统计，并快速跳转到建模、数据导入或查询页面。 @@ -100,7 +122,7 @@ Compose 中的 `server-data` 与 `hubble-data` 卷分别持久化保存图数据 若需新建图，可在页面点击新建按钮，填写图名称、可选别名并选择 Schema 模板或示例数据。连接信息由 Hubble 后端统一配置， 无需重复填写 Server 主机与账号。用户自定义 Schema 模板属于 PD 模式功能，详见 [分布式补充](/cn/docs/quickstart/toolchain/visualization/hugegraph-hubble-hstore/)。 -## Schema 建模：定义数据的形状 +## 定义 Schema {#schema-建模定义数据的形状} Schema 规定了图中允许存储的属性、顶点与边类型，并决定顶点 ID 的生成策略与索引规则。 在【元数据配置】中，可以通过列表视图维护定义，或通过图关系视图直观查看顶点与边的拓扑结构。 @@ -112,22 +134,22 @@ Schema 规定了图中允许存储的属性、顶点与边类型，并决定顶 | 边类型 | 起点和终点类型、属性、边的频次及排序键 | | 顶点索引 / 边索引 | 索引类型与字段，匹配实际过滤和范围查询 | -在人物与软件示例中：`person` 以 `name` 为主键生成 ID，`age` 与 `city` 允许为空； +在人物与软件示例中，`person` 以 `name` 为主键生成 ID，`age` 与 `city` 允许为空； `software` 采用自定义数值 ID；边类型 `created` 连接人物与软件。模型定义与 [Loader 示例](/cn/docs/quickstart/toolchain/hugegraph-loader/) 保持一致。 自主建模时，建议遵循 **属性 → 顶点类型 → 边类型 → 索引** 的顺序依次创建。 删除类型或创建/重建索引操作通常会提交为后台异步任务，操作提交后可在【异步任务】中查看最终执行结果。 -## GQL 工作台：查询与探索人物关系 +## 在 GQL 工作台查询人物关系 {#gql-工作台查询与探索人物关系} 进入【GQL 图遍历】，确认当前图为 `hugegraph`。工作台将查询编辑器与图展示区集成在一起， -支持即时执行与异步执行、常用语句收藏，并可随时从执行历史中重载语句。先在 Gremlin 编辑器中执行人物属性查询： +支持即时执行与异步执行、常用语句收藏，并可随时从执行历史中重载语句。先在 Gremlin 编辑器中查询人物属性。 ```groovy g.V().hasLabel('person').valueMap() ``` -这条查询适合在表格或 JSON 视图核对属性。要观察人物与软件的关系，执行： +这条查询适合在表格或 JSON 视图核对属性。也可以查询人物与软件的关系。 ```groovy g.V().hasLabel('person').outE('created').inV().path() @@ -142,29 +164,27 @@ Cypher 查询仅在 Server 后端支持时提供。快捷键会作用于当前 ### Text2GQL 自然语言查询 {#text2gql} -> [!NOTE] -> Text2GQL 后端能力（包括模型设置、内置/临时模型管理与 Prompt 模板）依赖包含 -> [apache/hugegraph-toolchain#824](https://github.com/apache/hugegraph-toolchain/pull/824) 的 Hubble 构建或后续发布版本；未合入该特性的旧版本仅提供预览占位，不包含模型设置页面。 - 在【自然语言】输入框中用自然语言描述查询需求，Text2GQL 会结合当前图的 Schema 自动生成只读 Gremlin 或 Cypher 语句并执行查询。 生成的语句支持二次编辑（执行前同样会进行只读安全校验）。切换至【异步】模式时，生成的查询与手写查询均可提交为后台异步任务。 输入框会自动恢复当前图最近一次的查询输入（不会自动触发执行），执行记录中也会完整保留原始自然语言问题。 ```gallery -![Text2GQL 查询与图结果](/cn/docs/images/hubble/text2gql.jpg) # 查询工作台 +![未配置模型时的自然语言入口与手写 Gremlin 查询结果](/cn/docs/images/hubble/text2gql.jpg) # 查询工作台，模型尚未配置，结果来自手写查询 ![内置模型设置](/cn/docs/images/hubble/text2gql-settings.jpg) # 模型设置 ![Gremlin 与 Cypher Prompt 模板](/cn/docs/images/hubble/text2gql-prompt.jpg) # Text2GQL Prompt ``` +配图中的模型尚未配置，图结果来自手写 Gremlin；自然语言生成功能需要先连接可用模型。 + | 模型来源 | 配置与保存方式 | | --- | --- | | 使用内置模型 | 在【系统与运维 → 模型设置】统一配置，Key 加密存入 H2，不回传浏览器 | | 使用临时模型 | 在查询页填写 API 类别、Base URL、模型名称和临时 API Key；Key 只保留在前端内存中，刷新后清除 | 使用 OpenAI、DeepSeek、GLM 等模型时选择【OpenAI 兼容接口】；使用本地运行的 Ollama 时选择【Ollama（本地）】（无认证时 Key 可留空）。 -注意：Base URL 必须能被 Hubble 后端服务正常访问（若 Hubble 运行在容器中，`localhost` 指向容器本身而非宿主机）。公网端点必须使用 HTTPS，HTTP 仅允许用于 +Base URL 必须能被 Hubble 后端服务正常访问（若 Hubble 运行在容器中，`localhost` 指向容器本身而非宿主机）。公网端点必须使用 HTTPS，HTTP 仅允许用于 本地回环或私网模型；在【模型设置】中修改 Base URL 时，必须重新填写 Key 或清空已保存的 Key。 -全局内置模型仅由 `admin` 账号（PD 模式下为超级管理员）统一配置；未开启 Server 认证时，任何访问者均可配置。 +全局内置模型由具备模型设置管理权限的管理员统一配置；未开启 Server 认证时，任何访问者均可配置。 在模型设置页面右上角打开 **Text2GQL Prompt**，可分别定制 Gremlin 与 Cypher 的生成规则及 Few-shot 示例。 点击【应用】后需保存模型设置，模板将持久化存入 H2 数据库供内置模型与临时模型共享，生成时会自动附加当前图的 Schema。 @@ -177,7 +197,7 @@ Cypher 查询仅在 Server 后端支持时提供。快捷键会作用于当前 页脚分别显示模型生成耗时与查询请求耗时；异步提交耗时仅代表任务入队耗时，不代表后台任务实际运行时间。 -## 内置图算法：用表单探索关系 +## 使用内置图算法 {#内置图算法用表单探索关系} 不想从头编写遍历语句时，可以进入【内置图算法】，搜索算法并填写参数。 邻居探索用于回答“这个点附近有什么”，路径算法用于寻找两个点的连接，相似度和排序算法用于比较或筛选顶点。 @@ -193,9 +213,9 @@ OLAP 批量算法需要 Computer 或 Vermeer 等外部计算环境。本例的 参数校验通过后点击卡片右侧运行按钮，或在参数表单内使用快捷键，即可观察一跳邻居。 ![内置邻居算法的参数与图结果](/cn/docs/images/hubble/algorithms.jpg) -{width="2033" height="977"} +{width="1600" height="1000"} -## 异步任务：确认后台操作的最终结果 +## 查看异步任务的最终结果 {#异步任务确认后台操作的最终结果} 【异步任务】集中显示当前图的后台任务，包括异步查询及部分 Schema、索引操作。 可以按任务类型和状态筛选，查看任务 ID、创建时间与执行状态；成功查询可打开结果，失败任务可展开错误信息。 @@ -204,13 +224,13 @@ OLAP 批量算法需要 Computer 或 Vermeer 等外部计算环境。本例的 例如把 `g.V().count()` 以异步方式提交后，先在列表确认成功，再查看返回的数量。 “提交成功”只表示请求被接受，不表示计算或索引已完成。排查失败时，结合任务错误和 Server 日志定位原因。 -## 数据源管理：把原始数据准备好 +## 准备数据源 {#数据源管理把原始数据准备好} 数据源负责“从哪里读、怎样解析”，可以被导入任务引用。Hubble 支持 FILE、HDFS、JDBC 和 Kafka； -不同来源需要对应的路径、连接或订阅配置。FILE 最适合先体验：上传文件后，配置格式、分隔符、编码和表头， +不同来源需要对应的路径、连接或订阅配置。首次体验可选择 FILE。上传文件后，配置格式、分隔符、编码和表头， 检查读取的列名，再进入映射。上传限制和允许的扩展名由 Hubble 配置决定。 -为人物示例保存一个 UTF-8 文件 `people.csv`： +为人物示例保存一个 UTF-8 文件 `people.csv`。 ```csv name,age,city @@ -221,10 +241,10 @@ docs_bob,32,Shanghai 创建 FILE 数据源并上传它，选择 CSV（逗号分隔、默认 UTF-8），列名为 `name,age,city`。 表头、分隔符和编码属于数据源配置，不在后面的映射步骤设置；源字段应与文件内容一致。 -## 数据导入：从字段映射到可查询的图 +## 映射字段并导入数据 {#数据导入从字段映射到可查询的图} 【数据导入】把数据源中的行转换为顶点和边。任务创建采用四段配置，先明确目标，再选择字段和映射，最后决定执行时机。 -沿用上面的数据源，在 `DEFAULT` / `hugegraph` 中创建人物导入任务： +沿用上面的数据源，在 `DEFAULT` / `hugegraph` 中创建人物导入任务。 1. **基础信息** 选择目标图与刚创建的数据源，填写任务名。 1. **源端字段** 选择 `name`、`age`、`city`，移到已选字段列表。 @@ -237,7 +257,7 @@ AUTOMATIC 由 Server 生成 ID，PRIMARY_KEY 根据映射的主键属性生成 I 任务列表用于管理配置和执行入口，详情中的执行历史用于查看每次实例的状态、导入量与错误。 Hubble 还提供周期调度和 Kafka 实时执行，调度方式需与数据源类型匹配。 -完成后返回 GQL 工作台验证： +完成后返回 GQL 工作台查询新导入的数据。 ```groovy g.V().hasLabel('person').has('name', within('docs_alice', 'docs_bob')).valueMap() @@ -263,7 +283,7 @@ Hubble 使用 Server 的认证和账号，没有独立用户库。匿名访问 连接支持默认角色接口的 Server 后，账号与空间访问可以分别管理。管理员管理全局账号， 空间管理员只管理已授权空间的成员；普通成员不会获得账号管理或运维入口。 -以下四种预设属于 PD 模式，不表示旧版本或单机 Server 都提供同样的权限编辑能力： +以下四种预设属于 PD 模式，不表示旧版本或单机 Server 都提供同样的权限编辑能力。 | 预设 | 范围与用途 | |---|---| @@ -278,10 +298,10 @@ Hubble 使用 Server 的认证和账号，没有独立用户库。匿名访问 旧 Server 未提供相应接口时，页面按能力隐藏或禁用操作，不应只凭按钮是否可见推断资源权限。 更完整的空间管理见 [分布式补充](/cn/docs/quickstart/toolchain/visualization/hugegraph-hubble-hstore/)。 -![认证模式下的账号列表（PD 模式示例）](/cn/docs/images/hubble/accounts.jpg) -{width="2033" height="977"} +![认证模式下的账号列表（单机模式示例）](/cn/docs/images/hubble/accounts.jpg) +{width="1600" height="1000"} -## 系统运维：检查单机 Server +## 检查单机 Server {#系统运维检查单机-server} 单机模式的【系统运维】提供【节点信息】，只显示 Server，不显示 PD / Store 节点或集群概览。 节点列表可搜索、筛选健康状态并进入详情；详情用于查看可用的版本、系统、JVM、Server 后端等指标。 @@ -319,7 +339,7 @@ FILE 上传默认允许 `csv,txt`，单文件 1 GB、总量 10 GB；需要覆盖 ## 停止试用环境或从源码构建 -不再需要示例时，在主仓库的 `docker/` 目录执行： +清理测试环境时，在 `docker/` 目录下执行以下命令： ```bash docker compose -p "${HUBBLE_DEMO_PROJECT:?}" -f docker-compose.yml down --volumes @@ -328,19 +348,21 @@ docker compose -p "${HUBBLE_DEMO_PROJECT:?}" -f docker-compose.yml down --volume 这会删除该项目的容器、网络、命名卷与匿名卷，示例图和 Hubble 导入任务也会丢失。 若想保留数据，下线时省略 `--volumes`，以后用同一项目名启动。 -需要与 master 精确一致的产物时，使用 JDK 17 和 Maven 从 Toolchain 构建。 -Maven 插件会安装所需的 Node/Yarn，无需预先安装；以下命令不执行测试： +从当前 master 构建时，需要 **JDK 17、Maven 3.9+、Python 3.9+、稳定版 Node.js 24（至少 24.22.0）**， +并通过 Corepack 使用仓库锁定的 **pnpm 12.14.0**。Maven 不再自动安装 Node/Yarn。 -```bash -git clone --branch master --single-branch https://github.com/apache/hugegraph-toolchain.git -cd hugegraph-toolchain -mvn install -pl hugegraph-client,hugegraph-loader -am -Dmaven.javadoc.skip=true -DskipTests -ntp -cd hugegraph-hubble -mvn package -Dmaven.javadoc.skip=true -DskipTests -ntp -cd apache-hugegraph-hubble-* -# 编辑 conf/hugegraph-hubble.properties，设置正确的 Server URL -bin/start-hubble.sh -``` +1. 按 [Java 17 构建说明](https://github.com/apache/hugegraph-toolchain/blob/master/docs/java17-migration.md) + 编译并安装 Server SDK 到独立 Maven 仓库。 +1. 按 [Hubble 源码构建步骤](https://github.com/apache/hugegraph-toolchain/blob/master/hugegraph-hubble/README.md) + 安装 Python 依赖，再使用同一 Maven 仓库构建 Client、Loader 和 Hubble（跳过测试）。 +1. 进入构建生成的 `hugegraph-hubble/apache-hugegraph-hubble-*` 目录，配置 `conf/hugegraph-hubble.properties` 中的 Server 连接信息后启动： + + ```bash + bin/start-hubble.sh -d + ``` +{.steps} + +当前 master 使用新的 H2 元数据库。升级时保留旧 Hubble 包及其数据，新版使用独立的数据目录；旧元数据不支持自动迁移，不能直接复用。 打包配置默认绑定本机 `localhost:8088`。`bin/stop-hubble.sh` 会先请求正常停机，超时后才强制终止。 需要开发与测试说明时，参考 [Toolchain 本地测试指南](/cn/docs/guides/toolchain-local-test/)。 diff --git a/content/cn/docs/quickstart/toolchain/hugegraph-loader.md b/content/cn/docs/quickstart/toolchain/hugegraph-loader.md index f1213cace8..881c6cf545 100644 --- a/content/cn/docs/quickstart/toolchain/hugegraph-loader.md +++ b/content/cn/docs/quickstart/toolchain/hugegraph-loader.md @@ -6,7 +6,105 @@ search_keywords: [HugeGraph Loader, 批量导入, 数据导入] search_boost: 1.6 --- -本文以 Toolchain `master`（当前版本为 `1.8.0`）为准。发布版和 Docker `latest` 可能与源码不同；使用时请核对版本，未发布的版本请按下文从源码构建。 +本文以 Toolchain `master` 分支（以 1.8 为基线，尚未正式发布）为准。发布版和 Docker `latest` 可能存在部分差异，使用未发布功能前请先核对版本。 + +### 快速上手：本地文件导入 {#quick-local-file-import} + +下面通过一个最简示例体验 Loader 的基本导入流程：导入两个人和一条关系，并在 Hubble 中查询验证。 +请先[安装 Loader](#2-获取-hugegraph-loader) 并启动 [HugeGraph Server](/cn/docs/quickstart/hugegraph/hugegraph-server/)。 +以下命令均在 Loader 安装目录下执行，连接本地默认图 `hugegraph`。若 Server 已启用鉴权，在命令中追加 `--username` 与 `--password`（或 `--token`）。 + +1. **创建数据文件** + + 两份 CSV 均不含表头，列名由下一步的映射文件指定。 + + ```bash + mkdir -p demo-loader + cat > demo-loader/people.csv <<'EOF' + docs-smoke-alice,29 + docs-smoke-bob,31 + EOF + cat > demo-loader/knows.csv <<'EOF' + docs-smoke-alice,docs-smoke-bob + EOF + ``` + +1. **创建 Schema 和映射文件** + + 本例使用独立的 `docs_*` 名称，避免与内置的 `person` 数据冲突。 + 映射文件中的路径相对于运行 Loader 时的工作目录。 + + ```bash + cat > demo-loader/schema.groovy <<'EOF' + schema.propertyKey("docs_name").asText().ifNotExist().create() + schema.propertyKey("docs_age").asInt().ifNotExist().create() + schema.vertexLabel("docs_smoke_person").properties("docs_name", "docs_age") + .primaryKeys("docs_name").ifNotExist().create() + schema.edgeLabel("docs_smoke_knows").sourceLabel("docs_smoke_person") + .targetLabel("docs_smoke_person").ifNotExist().create() + EOF + cat > demo-loader/struct.json <<'EOF' + { + "version": "2.0", + "structs": [ + { + "id": "people", + "input": { + "type": "FILE", + "path": "demo-loader/people.csv", + "format": "CSV", + "header": ["docs_name", "docs_age"] + }, + "vertices": [{"label": "docs_smoke_person"}] + }, + { + "id": "knows", + "input": { + "type": "FILE", + "path": "demo-loader/knows.csv", + "format": "CSV", + "header": ["source_name", "target_name"] + }, + "edges": [{ + "label": "docs_smoke_knows", + "source": ["source_name"], + "target": ["target_name"], + "field_mapping": {"source_name": "docs_name", "target_name": "docs_name"} + }] + } + ] + } + EOF + ``` + +1. **执行导入** + + 保持在同一工作目录。Server 运行在其他地址时，修改主机和端口。 + + ```bash + bin/hugegraph-loader.sh \ + --graphspace DEFAULT --graph hugegraph \ + --file demo-loader/struct.json --schema demo-loader/schema.groovy \ + --host 127.0.0.1 --port 8080 + ``` + + 首次成功导入后，统计应显示两个顶点、一条边。检查汇总中的成功数和失败数。 + +1. **在 Hubble 中查询结果** + + 将 Hubble 连接到同一个 Server 和图，在 Gremlin 工作台分别执行以下查询。 + + ```groovy + g.V().hasLabel('docs_smoke_person').valueMap('docs_name', 'docs_age') + g.V().has('docs_smoke_person', 'docs_name', 'docs-smoke-alice') + .out('docs_smoke_knows').values('docs_name') + ``` + + 第一条查询返回 Alice 的年龄 `29`、Bob 的年龄 `31`，第二条返回 `docs-smoke-bob`。 +{.steps} + +导入其他数据源时，继续阅读[映射文件参考](#33-编写数据源映射文件)。 +[内置完整示例](#4-完整示例)还包含软件顶点和边属性。 ### 1 HugeGraph-Loader 概述 @@ -22,15 +120,11 @@ HugeGraph-Loader 是 HugeGraph 的数据导入组件，能够将多种数据源 本地磁盘文件和 HDFS 文件支持断点续传。 -后面会具体说明。 - > **注意**：使用 HugeGraph-Loader 需要依赖 HugeGraph Server 服务，下载和启动 Server 请参考 [HugeGraph-Server 快速上手](/cn/docs/quickstart/hugegraph/hugegraph-server) > > **测试指南**：如需在本地运行 Loader 测试，请参考 [工具链本地测试指南](/cn/docs/guides/toolchain-local-test) -> **并发导入保护方案**：自 1.8.0 起，普通导入支持在同一 Server 实例上获取 owner 保护 -> （需配合返回 `protocol: 1` 的新版 Server 与对应 Loader；当前 `master` 分支暂未包含该协议）。 -> 旧版 Server 保持下文说明的原有导入行为。 +> **并发导入保护**：Toolchain master 分支的 Loader 已支持针对同一 Server 实例的 LOADING owner 保护。当配套 Server 的 `GET .../mode/owners` 返回 `protocol: 1` 时会自动生效；若 Server 无此接口，则保持原有导入行为。 ### 2 获取 HugeGraph-Loader @@ -83,25 +177,23 @@ services: #### 2.3 克隆源码编译安装 -克隆 `master` 分支： +从 master 分支编译 Loader 需要 Java 17、Maven 3.6.3+ 与 Python 3.9+。直接克隆仓库并编译： ```bash git clone --branch master --single-branch https://github.com/apache/hugegraph-toolchain.git -``` - -编译生成 tar 包： - -```bash cd hugegraph-toolchain -mvn clean package -pl hugegraph-loader -am -DskipTests -ntp +mvn package -pl hugegraph-client,hugegraph-loader -am -DskipTests -ntp +cd hugegraph-loader/apache-hugegraph-loader-* ``` +如需搭配开发阶段的 Server master SDK 编译，可参阅 [Java 17 迁移指引](https://github.com/apache/hugegraph-toolchain/blob/master/docs/java17-migration.md)。 + 如需读取 Oracle 数据，按所用 Oracle/JDK 版本准备兼容的 JDBC 驱动 JAR，并放入解压后 Loader 的 `lib/` 目录。 启动脚本会加载该目录中的 JAR；仅将驱动安装到 Maven 本地仓库并不会让运行中的 Loader 找到它。 ### 3 使用流程 -使用 HugeGraph-Loader 的基本流程分为以下几步： +导入数据前先准备图模型、数据文件和映射文件，再运行 Loader。 - 编写图模型 - 准备数据文件 @@ -110,9 +202,10 @@ mvn clean package -pl hugegraph-loader -am -DskipTests -ntp #### 3.1 编写图模型 -这一步是建模的过程，用户需要对自己已有的数据和想要创建的图模型有一个清晰的构想，然后编写 schema 建立图模型。 +先确定数据对应的顶点类型、边类型和属性，再编写 Schema。 -比如想创建一个拥有两类顶点及两类边的图，顶点是"人"和"软件"，边是"人认识人"和"人创造软件"，并且这些顶点和边都带有一些属性，比如顶点"人"有："姓名"、"年龄"等属性， "软件"有："名字"、"售卖价格"等属性；边"认识"有："日期"属性等。 +以下示例包含 `person` 和 `software` 两类顶点，以及 `knows` 和 `created` 两类边。 +人有姓名、年龄和城市，软件有名称和价格，两类边都记录日期。
由“认识”和“创建”边连接人物与软件顶点的示例图 @@ -120,7 +213,7 @@ mvn clean package -pl hugegraph-loader -am -DskipTests -ntp
-在设计好了图模型之后，我们可以用`groovy`编写出`schema`的定义，并保存至文件中，这里命名为`schema.groovy`。 +将 Schema 定义保存到 `schema.groovy`。 ```groovy // 创建一些属性 @@ -201,9 +294,10 @@ JSON 文件要求每一行都是一个 JSON 串，且每行的格式需保持一 loader 还支持以部分关系型数据库作为数据源，目前支持 MySQL、PostgreSQL、Oracle 和 SQL Server。 -但目前对表结构要求较为严格，如果导入过程中需要做**关联查询**，这样的表结构是不允许的。关联查询的意思是：在读到表的某行后，发现某列的值不能直接使用（比如外键），需要再去做一次查询才能确定该列的真实值。 +Loader 解析一行数据时，不会根据外键再查询其他表。输入行必须直接提供构造图元素所需的值。 +需要关联数据时，先在数据源侧准备好查询结果。 -举个例子：假设有三张表，person、software 和 created +以下以 `person`、`software`、`created` 三张表为例。 ```text // person 表结构 @@ -220,12 +314,13 @@ id | name | lang | price id | p_id | s_id | date ``` -如果在建模（schema）时指定 person 或 software 的 id 策略是 PRIMARY_KEY，选择以 name 作为 primary keys（注意：这是 hugegraph 中 vertexlabel 的概念），在导入边数据时，由于需要拼接出源顶点和目标顶点的 id，必须拿着 p_id/s_id 去 person/software 表中查到对应的 name，这种需要做额外查询的表结构的情况，loader 暂时是不支持的。这时可以采用以下两种方式替代： +若 `person` 和 `software` 使用 `PRIMARY_KEY` 策略，以 `name` 为主键，导入边时就需要两个端点的 `name`。 +`created` 表只有 `p_id` 和 `s_id`，Loader 不会用它们回查名称。可以改用以下两种 ID 设计。 -1. 仍然指定 person 和 software 的 id 策略为 PRIMARY_KEY，但是以 person 表和 software 表的 id 列作为顶点的主键属性，这样导入边时直接使用 p_id 和 s_id 和顶点的 label 拼接就能生成 id 了； -2. 指定 person 和 software 的 id 策略为 CUSTOMIZE，然后直接以 person 表和 software 表的 id 列作为顶点 id，这样导入边时直接使用 p_id 和 s_id 即可； +1. 保留 `PRIMARY_KEY` 策略，将两张顶点表的 `id` 列作为主键属性。Loader 可用 `p_id`、`s_id` 和顶点标签生成端点 ID。 +2. 使用自定义 ID 策略，将两张顶点表的 `id` 列直接作为顶点 ID，导入边时直接使用 `p_id` 和 `s_id`。 -关键点就是要让边能直接使用 p_id 和 s_id，不要再去查一次。 +这两种设计都让边数据直接提供端点 ID，省去回查。 ##### 3.2.2 准备顶点和边数据 @@ -270,13 +365,11 @@ Office,388 ##### 3.3.1 映射文件概述 -输入源的映射文件用于描述如何将输入源数据与图的顶点类型/边类型建立映射关系，以`JSON`格式组织，由多个映射块组成，其中每一个映射块都负责将一个输入源映射为顶点和边。 +映射文件使用 JSON 格式。每个 `structs` 映射块包含一个 `input`，以及该输入对应的 `vertices` 和 `edges` 映射。 -具体而言，每个映射块包含**一个输入源**和多个**顶点映射**与**边映射**块，输入源块对应上面介绍的`本地磁盘文件或目录`、`HDFS 文件或目录`和`关系型数据库`，负责描述数据源的基本信息，比如数据在哪，是什么格式的，分隔符是什么等。顶点映射/边映射与该输入源绑定，可以选择输入源的哪些列，哪些列作为 id、哪些列作为属性，以及每一列映射成什么属性，列的值映射成属性的什么值等等。 +`input` 描述数据位置、格式和分隔符。顶点和边映射指定标签、ID、属性及字段转换。 -以最通俗的话讲，每一个映射块描述了：要导入的文件在哪，文件的每一行要作为哪一类顶点/边，文件的哪些列是需要导入的，以及这些列对应顶点/边的什么属性等。 - -> 注意：0.11.0 版本以前的映射文件与 0.11.0 以后的格式变化较大，为表述方便，下面称 0.11.0 以前的映射文件（格式）为 1.0 版本，0.11.0 以后的为 2.0 版本。并且若无特殊说明，“映射文件”表示的是 2.0 版本的。 +> Loader 0.11.0 起使用 2.0 映射格式，此前的格式称为 1.0。下文未特别说明时，均使用 2.0 格式。 > [!DETAILS]- 点击展开/折叠 2.0 版本的映射文件的框架 @@ -936,61 +1029,12 @@ bin/hugegraph-loader.sh -g {GRAPH_NAME} -f ${INPUT_DESC_FILE} -s ${SCHEMA_FILE} #### 3.4.5 最小本地文件导入示例 -先确保 HugeGraph Server 已启动，当前用户可以访问 `DEFAULT` 图空间中的 `hugegraph` 图，并有权限创建 Schema 和写入数据。若 Server 已启用认证，在命令中增加对应的 `--username`、`--password` 或 `--token`。本例只需本地文件，不依赖 Kafka、HDFS 或 JDBC。 - -进入解压后的 Loader 安装目录，再创建数据、Schema 和 2.0 格式的映射文件： - -```bash -mkdir -p demo-loader - -cat > demo-loader/people.csv <<'EOF' -docs-smoke-alice,29 -docs-smoke-bob,31 -EOF - -cat > demo-loader/schema.groovy <<'EOF' -schema.propertyKey("docs_name").asText().ifNotExist().create() -schema.propertyKey("docs_age").asInt().ifNotExist().create() -schema.vertexLabel("docs_smoke_person").properties("docs_name", "docs_age").primaryKeys("docs_name").ifNotExist().create() -EOF - -cat > demo-loader/struct.json <<'EOF' -{ - "version": "2.0", - "structs": [ - { - "id": "people", - "input": { - "type": "FILE", - "path": "demo-loader/people.csv", - "format": "CSV", - "header": ["docs_name", "docs_age"] - }, - "vertices": [ - {"label": "docs_smoke_person"} - ], - "edges": [] - } - ] -} -EOF -``` - -`people.csv` 没有表头行，列名由映射文件的 `header` 提供。映射文件中的本地路径相对于运行命令时的工作目录。仍在 Loader 安装目录中执行导入；如果 Server 在其他容器中，请把 `--host` 改为 Loader 能访问到的 Server 地址。 - -```bash -bin/hugegraph-loader.sh \ - --graphspace DEFAULT --graph hugegraph \ - --file demo-loader/struct.json --schema demo-loader/schema.groovy \ - --host 127.0.0.1 --port 8080 -``` - -本例用独立的 `docs_smoke_person` 标签和 `docs_*` 属性，避免与预置的 `person` 样例数据冲突。成功时统计中应显示顶点写入成功数为 `2`、边写入成功数为 `0`。也可以在 Hubble 的 Gremlin 工作台查询 `g.V().hasLabel('docs_smoke_person').values('docs_name')`，结果应包含 `docs-smoke-alice` 和 `docs-smoke-bob`。 +按本页开头的[本地文件导入步骤](#quick-local-file-import)操作。 #### 3.4.6 并发导入与 LOADING owner {#loading-ownership} -自 1.8.0 版本起，Loader 支持并发导入模式所有权保护协议 -（需配合返回 `protocol: 1` 的新版 Server 与对应 Loader；当前 `master` 分支暂未包含该协议）。 +在 master 分支中，Loader 为普通导入实现了 LOADING 所有权管理协议，与 Server 的 `/mode/owners` 接口协同工作。 +若部署的 Server 提供该接口并返回 `protocol: 1`，Loader 会自动启用所有权保护；已发布的 Server 1.7.0 及更早版本则继续使用原有模式切换流程。 ```mermaid graph TD @@ -1296,52 +1340,50 @@ sh bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/f 执行的结果如 [4.5.1](#451-使用-docker-exec-直接导入数据) 所示 #### 4.6 使用 spark-loader 导入 -> Spark 版本：Spark 3+，其他版本未测试。当前源码使用 Spark 3.2.2 和 Scala 2.12；其他组合需自行验证。 -> -`spark-loader` 的参数分为两部分，注意：因二者参数名缩写存在重合部分，请使用参数全称。两种参数之间无需保证先后顺序。 -- hugegraph 参数（参考：[hugegraph-loader 参数说明](https://hugegraph.apache.org/cn/docs/quickstart/toolchain/hugegraph-loader/#341-%E5%8F%82%E6%95%B0%E8%AF%B4%E6%98%8E) ） -- Spark 任务提交参数（参考：[Submitting Applications](https://spark.apache.org/docs/3.3.0/submitting-applications.html#content)） -示例： -```bash -sh bin/hugegraph-spark-loader.sh --master yarn \ ---deploy-mode cluster --name spark-hugegraph-loader --file ./hugegraph.json \ ---username admin --token admin --host xx.xx.xx.xx --port 8093 \ ---graph graph-test --num-executors 6 --executor-cores 16 --executor-memory 15g -``` - -`bin/hugegraph-spark-loader.sh` 通过 `${SPARK_HOME}/bin/spark-submit` 提交 `org.apache.hugegraph.loader.spark.HugeGraphSparkLoader`，因此 `SPARK_HOME` 必须指向一个 Spark 安装目录。`lib/` 下的所有 jar 都会加入 classpath。Spark 应用名默认为 `hugegraph-spark-loader`，可以通过 `APP_NAME` 环境变量修改。 +当前 master 分支的 API 写入路径使用 Spark `3.5.8`、Scala `2.12`，driver 和 executor 均需 Java 17。 +POM 中的 Spark `3.2.2` 只提供编译期 API，不是运行引擎。设置 `SPARK_HOME` 和 `JAVA_HOME`， +再使用发行目录中的启动脚本和 shaded Loader JAR。 -`bin/get-params.sh` 负责拆分命令行：只有下列参数会交给 loader，其余参数原样传给 `spark-submit`。该拆分只识别参数全称，因此 `-f`、`-g` 这类缩写不会被识别。 +先创建 Schema，确保 executor 可以读取数据文件。CSV、TEXT 数据须使用 UTF-8、去掉表头行，并在映射中显式指定 `header`。 +其他字符集和 `has_header: true` 会被拒绝。边依赖已有顶点时，先导入顶点。将以下路径替换为准备好的顶点映射文件。 -```text ---graph --schema --host --port --username --token --protocol ---trust-store-file --trust-store-password --clear-all-data --clear-timeout ---incremental-mode --failure-mode --batch-insert-threads --single-insert-threads ---max-conn --max-conn-per-route --batch-size --max-parse-errors --max-insert-errors ---timeout --shutdown-timeout --retry-times --retry-interval --check-vertex ---print-progress --dry-run --sink-type --vertex-partitions --edge-partitions --help +```bash +bin/hugegraph-spark-loader.sh \ + --master 'local[2]' --driver-memory 1g \ + --file /path/to/vertex-mapping.json \ + --host 127.0.0.1 --port 8080 --graph hugegraph --sink-type true ``` -`--file` 单独处理：使用 `--deploy-mode cluster` 时，映射文件会通过 `--files` 分发到 executor，loader 只收到它的文件名；其他情况下路径原样传入。 +启动脚本可同时接收 `--master`、`--conf`、`--jars` 等 Spark 参数和 Loader 参数。 +额外的 JDBC 驱动需通过 `--jars` 传入，仅放在 `lib/` 中不会分发到 executor。 +导入后查询目标图，核对数量和属性；API 写入路径的插入计数器不统计分布式写入。 -该模式下 loader 只读取 FILE、HDFS 和 JDBC 输入源，KAFKA 和 GRAPH 输入源会被拒绝。 - -默认情况下（`--sink-type true`），每个 Spark 分区各自创建一个 HugeClient，通过 HugeGraph server API 写入顶点和边。使用 `--sink-type false` 时，loader 会生成 HFile 并直接 bulkload 到 HBase，此时表名和 ZooKeeper 配置取自 `--vertex-table-name`、`--edge-table-name`、`--hbase-zk-quorum`、`--hbase-zk-port`、`--hbase-zk-parent`、`--vertex-partitions` 和 `--edge-partitions`。 +该路径拒绝 `--incremental-mode true` 和 `--failure-mode true`。Standalone Spark 仅支持 client 部署模式。 +Spark 3.5.8 / Java 17 不支持 HBase bulkload（`--sink-type false`）。已有 bulkload 部署建议继续沿用原环境验证通过的 +Loader 发布包及其相关依赖。更多细节见 [Spark 迁移说明](https://github.com/apache/hugegraph-toolchain/blob/master/hugegraph-loader/docs/spark-java17.md)。 #### 4.7 使用 flink-cdc-loader 导入 -> 当前源码使用 Flink 1.13.5、flink-connector-mysql-cdc 2.2.1 和 Scala 2.12；其他组合需自行验证。 - -`bin/hugegraph-flinkcdc-loader.sh` 通过 `${FLINK_HOME}/bin/flink run` 提交 `org.apache.hugegraph.loader.flink.HugeGraphFlinkCDCLoader`，因此必须设置 `FLINK_HOME`。该任务用 Flink CDC 捕获 MySQL 的变更事件并应用到图上，从而让图与源表保持同步。 +当前 master 分支适配 Flink `2.2.1`、MySQL CDC `3.6.0-2.2` 和 Java 17。 +此前使用 Flink 1.13.5、CDC 2.2.1 的示例属于旧实现。 +设置 `FLINK_HOME`，确保提交进程、JobManager 和 TaskManager 都使用 Java 17。 +保留 Flink 的模块访问参数，发行目录的 `lib/` 中须恰好有一个 shaded Loader JAR。 -映射文件格式与命令行 loader 相同，但每个输入源都必须是 MySQL 的 JDBC 输入源：loader 从中读取 `url`、`database`、`table`、`username` 和 `password`，并从 `url` 解析出主机和端口。顶点和边映射的用法不变。[3.4.1](#341-参数说明) 中的 `--cdc-flush-interval` 和 `--cdc-sink-parallelism` 只在该模式下生效。 +每个映射都需使用 MySQL JDBC 输入，包含 `url`、`database`、`table`、`username` 和 `password`。 +MySQL 须配置 `binlog_row_image=FULL`。UPDATE 事件需要显式指定非空的 `update_strategies`，且图元素标识保持不变。 +顶点 ID、边端点或排序键变化时，任务会拒绝更新。CDC 也会拒绝 `--dry-run true` 和 `--use-prefilter true`。 -命令行同样由 `bin/get-params.sh` 按 4.6 的方式拆分，非 loader 参数会传给 `flink run`。 - -示例： +不带分隔符时，所有参数均交给 Loader。需要设置 Flink 参数时，将其放在 `--` 之前，Loader 参数放在之后。 +准备好 `mysql-cdc.json` 映射后，使用单个 sink writer 提交任务。 ```bash -sh bin/hugegraph-flinkcdc-loader.sh --file ./mysql-cdc.json \ ---host xx.xx.xx.xx --port 8080 --graph hugegraph --username admin --token admin +bin/hugegraph-flinkcdc-loader.sh -p 1 -- \ + --file ./mysql-cdc.json --host 127.0.0.1 --port 8080 \ + --graph hugegraph --cdc-sink-parallelism 1 ``` + +每个映射使用一个 source reader 和一个 sink writer。`--cdc-sink-parallelism` 默认为 `1`，其他取值会被拒绝。 +`--` 之后的 Loader 参数 `-p` 表示 Server 端口。Checkpoint 刷新不提供图写入事务，也不保证 exactly-once。 +在生产环境部署边 CDC 或更复杂的多表映射前，建议先在真实环境中充分验证。 +支持范围和恢复流程见 [Flink CDC 说明](https://github.com/apache/hugegraph-toolchain/blob/master/hugegraph-loader/docs/flink-cdc-java17.md)。 diff --git a/content/cn/docs/quickstart/toolchain/hugegraph-tools.md b/content/cn/docs/quickstart/toolchain/hugegraph-tools.md index 40750ca7c7..93301682a6 100644 --- a/content/cn/docs/quickstart/toolchain/hugegraph-tools.md +++ b/content/cn/docs/quickstart/toolchain/hugegraph-tools.md @@ -6,7 +6,7 @@ weight: 3 ### 1 HugeGraph-Tools概述 -HugeGraph-Tools 是 HugeGraph 的自动化部署、管理和备份/还原组件。 +HugeGraph-Tools 提供图与任务管理、备份和恢复命令，也保留旧版 Server/Studio 的部署命令。 > **测试指南**：如需在本地运行 Tools 测试，请参考 [工具链本地测试指南](/cn/docs/guides/toolchain-local-test) @@ -19,7 +19,7 @@ HugeGraph-Tools 包含在 Toolchain 发布包中，可以下载发布包，也 #### 2.1 下载二进制tar包 -下载最新版本的 HugeGraph-Toolchain 包, 然后进入 tools 子目录 +以下以已发布的 HugeGraph-Toolchain `1.7.0` 为例。其他发布版本见[下载页](/cn/docs/download/download/)。 ```bash export VERSION=1.7.0 @@ -32,42 +32,83 @@ cd "${ARCHIVE}/apache-hugegraph-tools-incubating-${VERSION}" ``` #### 2.2 下载源码编译安装 -源码编译前请确保安装了wget命令 -下载最新版本的 HugeGraph-Toolchain 源码包, 然后根目录编译或者单独编译 tool 子模块： +任选一种方式构建发布版 `1.7.0` Tools，开始前先安装 Maven 和 JDK 11。源码包方式还需要 `wget`。 +两段命令均包含编译和进入生成目录，最后用 `bin/hugegraph help` 查看可用命令。 -```bash -# 1. get from github -git clone https://github.com/apache/hugegraph-toolchain.git +```bash {tab="Git 标签"} +git clone --branch 1.7.0 --single-branch https://github.com/apache/hugegraph-toolchain.git +cd hugegraph-toolchain +mvn package -pl hugegraph-tools -am -DskipTests -ntp +cd hugegraph-tools/apache-hugegraph-tools-incubating-1.7.0 +bin/hugegraph help +``` -# 2. 下载发布版源码包 +```bash {tab="源码包"} export VERSION=1.7.0 export ARCHIVE="apache-hugegraph-toolchain-incubating-${VERSION}" wget "https://downloads.apache.org/hugegraph/${VERSION}/${ARCHIVE}-src.tar.gz" +tar zxf "${ARCHIVE}-src.tar.gz" +cd "${ARCHIVE}-src" +mvn package -pl hugegraph-tools -am -DskipTests -ntp +cd "hugegraph-tools/apache-hugegraph-tools-incubating-${VERSION}" +bin/hugegraph help ``` -编译生成 tar 包: +编译还会在源码根目录下生成 `hugegraph-tools/target/apache-hugegraph-tools-incubating-1.7.0.tar.gz`。 -```bash -cd hugegraph-toolchain -mvn package -pl hugegraph-tools -am -DskipTests -ntp -``` +#### 2.3 构建当前源码 {#build-current-source} -生成的 tar 包位于 `hugegraph-tools/target/apache-hugegraph-tools-${version}.tar.gz`，同时会在 `hugegraph-tools/apache-hugegraph-tools-${version}` 生成解压后的目录（包含 `bin/` 和 `lib/`） +若需要使用 master 分支未发布的新特性，可按以下步骤从源码构建 Tools（当前 master 以 1.8 为基线，需 JDK 17、Maven 3.6.3+ 与 Python 3.9+）： +1. **获取 Toolchain 源码** + + ```bash + git clone --branch master --single-branch https://github.com/apache/hugegraph-toolchain.git + cd hugegraph-toolchain + ``` + +1. **构建配套 Server SDK** + + 执行以下脚本下载 Server 源码并将配套 SDK 安装到独立 Maven 仓库： + + ```bash + java -version + mvn -version + candidate_dir=$(mktemp -d) + mkdir -p "$candidate_dir/m2" + bash hugegraph-client/assembly/travis/install-candidate-sdk.sh \ + master "$candidate_dir/server" "$candidate_dir/m2" + ``` + + 详细说明见 [Java 17 迁移指引](https://github.com/apache/hugegraph-toolchain/blob/master/docs/java17-migration.md)。 + +1. **打包 Tools** + + ```bash + mvn -Dmaven.repo.local="$candidate_dir/m2" \ + -Dsdk.validation.mode=candidate \ + package -pl hugegraph-tools -am -DskipTests -Dmaven.javadoc.skip=true -ntp + cd hugegraph-tools/apache-hugegraph-tools-* + bin/hugegraph help + ``` +{.steps} + +使用这份源码构建时保留 SDK 仓库。master 分支生成目录名不含 `incubating`。 ### 3 使用 #### 3.1 功能概览 -解压后，进入 apache-hugegraph-tools-${version} 目录，可以使用`bin/hugegraph`或者`bin/hugegraph help`来查看 usage 信息，使用`bin/hugegraph help <子命令>`查看单个子命令的 usage。主要分为： +进入解压或编译生成的 Tools 目录，用 `bin/hugegraph help` 查看命令列表， +用 `bin/hugegraph help <子命令>` 查看单个命令的用法。可用命令分为以下几类。 - 图管理类，graph-mode-set、graph-mode-get、graph-list、graph-get、graph-clear、graph-create、graph-clone 和 graph-drop - 异步任务管理类，task-list、task-get、task-delete、task-cancel 和 task-clear - Gremlin类，gremlin-execute 和 gremlin-schedule - 备份/恢复类，backup、restore、migrate、schedule-backup 和 dump - 认证数据备份/恢复类，auth-backup 和 auth-restore -- 安装部署类，deploy、clear、start-all 和 stop-all +- 旧版 Server/Studio 部署，deploy、clear、start-all 和 stop-all ```bash Usage: hugegraph [options] [command] [command options] @@ -75,20 +116,21 @@ Usage: hugegraph [options] [command] [command options] ##### 3.2 [options]-全局变量 -`options`是 HugeGraph-Tools 的全局变量，可以在 hugegraph-tools/bin/hugegraph 中配置,包括： +以下全局选项可以通过命令行或环境变量设置。 -- --graph，HugeGraph-Tools 操作的图的名字，默认值是 hugegraph -- --url，HugeGraph-Server 的服务地址，默认是 http://127.0.0.1:8080 -- --user，当 HugeGraph-Server 开启认证时，传递用户名 -- --password，当 HugeGraph-Server 开启认证时，传递用户的密码 -- --timeout，连接 HugeGraph-Server 时的超时时间，默认是 30s -- --trust-store-file，证书文件的路径，当 --url 使用 https 时，HugeGraph-Client 使用的 truststore 文件，默认为空，代表使用 hugegraph-tools 内置的 truststore 文件 conf/hugegraph.truststore -- --trust-store-password，证书文件的密码，当 --url 使用 https 时，HugeGraph-Client 使用的 truststore 的密码，默认为空，代表使用 hugegraph-tools 内置的 truststore 文件的密码 -- --throw-mode，HugeGraph-Tools 出错时是否直接抛出异常，而不是打印错误信息后退出，默认为 false（主要用于测试） +- `--graph` 指定要操作的图，默认为 `hugegraph`。 +- `--url` 指定 Server 地址，默认为 `http://127.0.0.1:8080`。 +- `--user` 在 Server 启用认证时提供用户名。 +- `--password` 提供该用户的密码。 +- `--timeout` 设置连接超时时间，默认为 30 秒。 +- `--trust-store-file` 指定 HTTPS truststore，留空时使用内置的 `conf/hugegraph.truststore`。 +- `--trust-store-password` 提供其密码，留空时使用内置 truststore 的密码。 +- `--throw-mode` 默认为 `false`，出错时打印错误并退出。设为 `true` 时抛出异常，主要用于测试。 -> 连接协议由 --url 的 scheme 决定：使用 `https://...` 即通过 https 连接。--trust-store-file 和 --trust-store-password 只能在 --url 使用 https 时设置，--user 和 --password 必须同时提供或同时省略。 +> `--url` 的 scheme 决定连接协议。使用 `https://...` 连接 HTTPS 时，才可设置 truststore 选项。 +> `--user` 和 `--password` 必须同时提供或同时省略。 -上述全局变量，也可以通过环境变量来设置。一种方式是在命令行使用 export 设置临时环境变量，在该命令行关闭之前均有效 +使用 `export` 设置的环境变量对当前终端及其启动的命令生效。 | 全局变量 | 环境变量 | 示例 | @@ -114,7 +156,7 @@ export HUGEGRAPH_GRAPH=hugegraph ./bin/hugegraph gremlin-execute --script 'g.V().count()' ``` -另一种方式是在 bin/hugegraph 脚本中设置环境变量： +也可以在 `bin/hugegraph` 脚本中设置这些变量。 ```text #!/bin/bash @@ -259,6 +301,10 @@ export HUGEGRAPH_GRAPH=hugegraph ##### 3.8 安装部署类 +注意：此处的安装部署命令仅适用于早期历史版本（Server 0.6.1~0.10.4 与 Studio 0.6.1~0.10.0），不支持当前版本的 HugeGraph Server 或 Hubble。 +新部署请参考 [Server 快速上手](/cn/docs/quickstart/hugegraph/hugegraph-server/) 与 [Hubble 快速上手](/cn/docs/quickstart/toolchain/hugegraph-hubble/)， +以下说明仅供维护旧环境参考。 + - deploy，一键下载、安装和启动 HugeGraph-Server 和 HugeGraph-Studio - -v，必填项，指定要安装的 HugeGraph-Server 和 HugeGraph-Studio 版本，必须是 bin/version-map.yaml 中列出的版本之一（0.6、0.7、0.8、0.9、0.10），脚本据此映射到对应的 server 和 studio 发布版本 - -p，必填项，指定安装的 HugeGraph-Server 和 HugeGraph-Studio 目录 diff --git a/content/cn/docs/quickstart/toolchain/visualization/hugegraph-hubble-hstore.md b/content/cn/docs/quickstart/toolchain/visualization/hugegraph-hubble-hstore.md index 74cc8608e2..1ddf3034e1 100644 --- a/content/cn/docs/quickstart/toolchain/visualization/hugegraph-hubble-hstore.md +++ b/content/cn/docs/quickstart/toolchain/visualization/hugegraph-hubble-hstore.md @@ -17,11 +17,21 @@ search_keywords: [HugeGraph Hubble, HStore, PD, GraphSpace, 集群管理] Docker `latest` 为动态标签，配图用于定位界面，实际版本以部署镜像的 digest 及构建信息为准。 不同版本可提供的指标和权限接口有所区别。 +本文截图基于 HugeGraph Hubble 界面，中英文界面功能与操作一致。 + ## 连接分布式集群 Hubble 仍通过 **Server 的图 API** 管理数据。区别在于，分布式模式通过 PD 发现 Server， 同时从 PD 和 Store 获取集群信息；Hubble 不直接读写 Store 中的图数据。 +```mermaid +flowchart LR + H[Hubble] -->|图 API| S[Server] + S -->|读写图数据| T[Store] + H -.->|服务发现与集群信息| P[PD] + H -.->|节点指标| T +``` + 1. **生成 Hubble 配置。** 按 Docker README 在主仓库的 `docker/` 目录准备 `.env`。 ```bash @@ -45,7 +55,7 @@ Hubble 仍通过 **Server 的图 API** 管理数据。区别在于，分布式 若分别部署服务，参照 [PD 部署指南](/cn/docs/quickstart/hugegraph/hugegraph-pd/) 和 [HStore 部署指南](/cn/docs/quickstart/hugegraph/hugegraph-hstore/)；源码或二进制包启动的 Hubble 则编辑其 -`conf/hugegraph-hubble.properties`。以下连接项与官方最小拓扑一致，其他部署请使用 Hubble 后端实际可达的地址： +`conf/hugegraph-hubble.properties`。以下连接项与官方最小拓扑一致，其他部署请使用 Hubble 后端实际可达的地址。 ```properties pd.enabled=true @@ -61,10 +71,10 @@ pd.server=pd:8620 | `pd.peers` | PD **gRPC** 地址；多个地址用逗号分隔。 | `127.0.0.1:8686` | | `pd.server` | 集群运维使用的 PD **REST** 地址，不是 gRPC peer 列表。 | `127.0.0.1:8620` | -请注意区分两个端口：`8686` 为 gRPC 端口，`8620` 为 REST 端口，切勿混用；跨容器通信时请使用容器服务名或网络可达 IP，避免使用 `127.0.0.1`。 +请区分两个端口。`8686` 为 gRPC 端口，`8620` 为 REST 端口，切勿混用；跨容器通信时请使用容器服务名或网络可达 IP，避免使用 `127.0.0.1`。 发行包默认配置中显式设为 `pd.enabled=false`；若未配置该项，代码默认值为 `true`，因此单机与分布式部署均建议显式声明该配置。 -修改配置后：源码或二进制部署需重启 Hubble 服务；Docker Compose 部署则在 `docker/` 目录下重新创建 Hubble 容器以重新加载挂载配置 -（沿用启动时的项目名及所有 `-f` 参数）： +修改配置后，源码或二进制部署需重启 Hubble 服务；Docker Compose 部署则在 `docker/` 目录下重新创建 Hubble 容器以重新加载挂载配置 +（沿用启动时的项目名及所有 `-f` 参数）。 ```bash docker compose -f docker-compose-hstore.yml up -d --force-recreate hubble @@ -95,7 +105,7 @@ GraphSpace（图空间）将图、Schema 模板和访问权限组织在同一个 列表只显示账号可访问的空间；看不到预期空间时，先检查成员权限，不要反复新建图。 ![图空间创建表单与高级资源限制入口](/cn/docs/images/hubble/graphspace.jpg) -{width="2048" height="984"} +{width="1600" height="1000"} ### 复用 Schema 模板 @@ -103,7 +113,7 @@ GraphSpace（图空间）将图、Schema 模板和访问权限组织在同一个 当一组业务图需要同样的顶点类型、边类型和索引时，可以先保存模板，再在创建图时选择它，减少重复建模。 例如 `research` 中的用户模板用于该空间后续创建的图；切换到另一个空间后，应重新选择该空间的模板。 -用户模板与主篇的内置示例模板不同：内置模板用于快速体验预设模型，用户模板用于保存自己的模型并长期复用。 +主篇的内置示例模板用于快速体验预设模型。用户模板用于保存自己的模型并长期复用。 模板本身不等于导入数据；是否加载示例数据是创建图时的另一个选择。 开启 Server 认证时，创建用户模板需要所在空间的写权限，更新或删除还要求模板所有者或相应管理权限。 匿名模式不进行按用户的权限与模板所有者检查。 @@ -113,7 +123,7 @@ GraphSpace（图空间）将图、Schema 模板和访问权限组织在同一个 账号创建、登录和个人资料管理见[基础指南](/cn/docs/quickstart/toolchain/hugegraph-hubble/)； 分布式模式额外提供“管理 GraphSpace 成员”，在已有账号与指定空间之间分配权限。 -支持权限预设的 Server 提供以下常用选择： +支持权限预设的 Server 提供以下常用选择。 | 预设 | 适用对象 | 作用范围 | |---|---|---| @@ -128,7 +138,7 @@ GraphSpace（图空间）将图、Schema 模板和访问权限组织在同一个 认证配置见 [Server 认证与授权](/cn/docs/config/config-authentication/)。 ![GraphSpace 成员与权限预设](/cn/docs/images/hubble/members.jpg) -{width="2048" height="984"} +{width="1600" height="1000"} ## 从集群概览定位异常 @@ -136,7 +146,7 @@ GraphSpace（图空间）将图、Schema 模板和访问权限组织在同一个 概览中的 PD Leader、在线 Store 数、图数、分区数、副本数和数据量，帮助判断当前连接的规模及哪些节点需要关注。 分区和副本描述数据分布；数据量是采集到的用量，不是图空间配置的存储上限。 -先看集群和来源状态，再进入异常节点：`UP` 表示本次采集的来源正常；`DEGRADED` 表示集群或部分来源异常； +先看集群和来源状态，再进入异常节点。`UP` 表示本次采集的来源正常；`DEGRADED` 表示集群或部分来源异常； `DOWN` 表示相应探测未成功。拓扑可用而指标部分缺失时，仍可能正常查询图。 页面会区分不支持、不可用、数据过期和采集失败，并显示采集时间；空值不是零，历史值也不等于当前状态。 @@ -154,7 +164,7 @@ GraphSpace（图空间）将图、Schema 模板和访问权限组织在同一个 ![Store 节点的系统、磁盘、Raft 和分区指标](/cn/docs/images/hubble/store-node.jpg) # Store 详情，再检查节点指标 ``` -PD Leader 与 Store 的 Leader 分区不是同一概念：前者协调集群元数据，后者负责相应数据分区的 Raft 组。 +PD Leader 与 Store 的 Leader 分区分别负责不同的工作。前者协调集群元数据，后者负责相应数据分区的 Raft 组。 Hubble 展示上游提供的观测值，不代替完整的 Raft 副本一致性检查；各版本可用的指标不同。 开启 Server 认证时，集群运维读取要求超级管理员（`ADMIN` 级别），空间管理员和普通成员不会因此获得集群读取权限。 @@ -164,7 +174,7 @@ Hubble 展示上游提供的观测值，不代替完整的 Raft 副本一致性 PD/Store 的运维凭据由 Hubble **后端**使用，与浏览器登录 Server 的账号分开。 Compose 部署修改上文生成的宿主机 `docker/conf/hubble/hstore.local.properties`；其他部署修改 Hubble 包内配置文件。 -其中 PD 密码由脚本生成并与 `.env` 中的密钥保持一致，Store 启用认证时再配置对应服务账号。不要把密码写进文档、截图或提交的配置： +其中 PD 密码由脚本生成并与 `.env` 中的密钥保持一致，Store 启用认证时再配置对应服务账号。不要把密码写进文档、截图或提交的配置。 | 配置 | 随包默认值 | 设置方式 | |---|---|---| @@ -173,14 +183,14 @@ Compose 部署修改上文生成的宿主机 `docker/conf/hubble/hstore.local.pr | `operations.store.allowed_targets` | `[http://127.0.0.1:8520,http://[::1]:8520]` | 列出信任的 Store 指标来源。 | {.td-table--wide} -例如容器中的 Store 上报 `store:8520`，将白名单配置为： +例如容器中的 Store 上报 `store:8520`，将白名单配置为以下内容。 ```properties operations.store.allowed_targets=[http://store:8520] ``` 多节点集群中需列出所有允许访问的目标地址。每项必须是包含显式端口的 `http` 或 `https` 源地址（Origin），不可包含路径、凭据或通配符， -且需与 PD 返回的 Store 指标目标完全一致。注意：仅在白名单中添加地址不会自动建立节点发现。 +且需与 PD 返回的 Store 指标目标完全一致。仅在白名单中添加地址不会自动建立节点发现。 -若集群拓扑可见但节点指标不完整，请依次排查：Hubble 后端到 PD REST 的网络连通性与认证凭据、PD 返回的 Store REST 与指标地址， +若集群拓扑可见但节点指标不完整，请依次检查 Hubble 后端到 PD REST 的网络连通性与认证凭据、PD 返回的 Store REST 与指标地址， 以及目标地址是否已加入上述白名单。概览中的部分可用状态仅表示特定来源指标采集受阻，并不代表图 API 服务不可用。 diff --git a/content/en/docs/clients/gremlin-console.md b/content/en/docs/clients/gremlin-console.md index ad4846d9c3..6e3cf0d49d 100644 --- a/content/en/docs/clients/gremlin-console.md +++ b/content/en/docs/clients/gremlin-console.md @@ -69,73 +69,80 @@ For more Gremlin statements, please refer to [Tinkerpop Official Website](http:/ ### 2 Client/Server mode -Gremlin Console connects to HugeGraph Server through WebSocket. The default configuration uses `WsAndHttpChannelizer`, which handles both WebSocket and HTTP requests, so there is no need to switch the Channelizer. +Gremlin Console connects to HugeGraph Server through WebSocket. Use the Console and `conf/remote.yaml` bundled with the same Server version. +Keep the serializer settings from that file; change only the address and, if authentication is enabled, credentials. +Older configuration files may reference serializer classes that no longer exist. -```yaml -# vim conf/gremlin-server.yaml -# ...... -channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer -# ...... -``` +1. Prepare the connection configuration and start Server. -Confirm that `host` and `port` match the settings in `remote.yaml`, and then follow the [steps](/docs/quickstart/hugegraph/hugegraph-server) to start HugeGraph Server. + Check the `host` and `port` in `conf/gremlin-server.yaml`. The default `WsAndHttpChannelizer` handles both WebSocket and HTTP requests; + it does not need to be changed. In `conf/remote.yaml`, `hosts` must contain the Server address reachable from the Console, + and `port` must match the Gremlin port (default `8182`, not the REST port `8080`). + Start Server following the [quick start](/docs/quickstart/hugegraph/hugegraph-server). -Then enter Gremlin-Console: + The current master configuration uses GraphBinary V1 with HugeGraph's type registry: -```bash -> ./bin/gremlin-console.sh + ```yaml + hosts: [localhost] + port: 8182 + serializer: + className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1 + config: + serializeResultToString: false + builder: org.apache.hugegraph.io.HugeGraphTypeSerializerRegistryBuilder + ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry] + ``` - \,,,/ - (o o) ------oOOo-(3)-oOOo----- -plugin activated: HugeGraph -plugin activated: tinkerpop.server -plugin activated: tinkerpop.utilities -plugin activated: tinkerpop.tinkergraph -``` + If Server requires authentication, add your actual credentials to the same file: -To connect to the server, you need to specify the connection parameters in the configuration file, and there is a default `remote.yaml` file in the conf directory - -```yaml -# cat conf/remote.yaml -hosts: [localhost] -port: 8182 -serializer: { - className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, - config: { - serializeResultToString: false, - ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry] - } -} -``` + ```yaml + username: admin + password: "replace-with-your-password" + ``` -If the Server runs in auth mode, add the credentials to the same file: + The bundled `remote-objects.yaml` and `gremlin-driver-settings.yaml` use the same serializer settings. Keep them aligned if you use them. -```yaml -username: admin -password: pa -``` +1. Start Console from the Server distribution directory and create a client with a traversal alias. -The `conf` directory also ships `remote-objects.yaml` and `gremlin-driver-settings.yaml`, which carry the same host, port, and serializer settings. + ```bash + ./bin/gremlin-console.sh + ``` -```groovy -gremlin> :remote connect tinkerpop.server conf/remote.yaml -==>Configured localhost/127.0.0.1:8182 -``` + Server binds the graph `hugegraph` in graphspace `DEFAULT` as `DEFAULT-hugegraph`, and its traversal source as `__g_DEFAULT-hugegraph`. + Because hyphens are invalid in Groovy variable names, use `.alias(['g': '__g_DEFAULT-hugegraph'])` to map it to `g` when creating the driver client: + + ```groovy + import org.apache.tinkerpop.gremlin.driver.Cluster + cluster = Cluster.open('conf/remote.yaml') + client = cluster.connect().alias(['g': '__g_DEFAULT-hugegraph']) + ``` + +1. Submit queries through the client. + + ```groovy + client.submit('g.V().count()').all().get()[0].object + client.submit('g.V().toList().size()').all().get()[0].object + ``` + + Both return `6` if the sample graph was preloaded at Server startup (other graphs return their actual vertex count). + When submitting scripts via `client.submit()`, no `:>` prefix is needed. + +1. Close the client and cluster when finished. + + ```groovy + client.close() + cluster.close() + ``` +{.steps} -Server-side graphs are bound under a graphspace-qualified name, so the graph `hugegraph` in graphspace `DEFAULT` is bound as `DEFAULT-hugegraph` and its traversal source as `__g_DEFAULT-hugegraph`. A bare `hugegraph` does not resolve on the Server, and `DEFAULT-hugegraph` is not a valid Groovy identifier, so a remote script reaches the traversal source through an alias. If the sample graph was preloaded when HugeGraph-Server started, a query looks like this: +Alternatively, you can connect using Gremlin Console's built-in `:remote` command: ```groovy -gremlin> import org.apache.tinkerpop.gremlin.driver.Cluster -gremlin> cluster = Cluster.open('conf/remote.yaml') -gremlin> client = cluster.connect().alias(['g': '__g_DEFAULT-hugegraph']) -gremlin> client.submit('g.V().count()').all().get()[0].object -==>6 -gremlin> client.submit('g.V().toList().size()').all().get()[0].object -==>6 -gremlin> client.close(); cluster.close() +:remote connect tinkerpop.server conf/remote.yaml +// Use :> as a prefix when submitting scripts through the Console remote connection. +:remote close ``` -> NOTE: In Client/Server mode, all operations related to the Server should be prefixed with `:> `. If not added, it indicates local console operations. A `:> ` script carries no alias, so it can only use names the Server itself has bound. +When connected with `:remote`, expressions prefixed with `:>` are sent to the remote Server, while normal expressions execute locally in Console. -For more information on the use of Gremlin-Console, please refer to [Tinkerpop Official Website](http://tinkerpop.apache.org/docs/current/reference/) +For more information, see the [TinkerPop Gremlin Console reference](https://tinkerpop.apache.org/docs/current/reference/#gremlin-console). diff --git a/content/en/docs/config/config-authentication.md b/content/en/docs/config/config-authentication.md index 80b0db9951..0bf25a97cc 100644 --- a/content/en/docs/config/config-authentication.md +++ b/content/en/docs/config/config-authentication.md @@ -7,17 +7,15 @@ search_boost: 1.7 --- ### Overview -To facilitate authentication usage in different user scenarios, HugeGraph currently provides built-in authorization `StandardAuthenticator` mode, -which supports multi-user authentication and fine-grained access control. It adopts a 4-layer design based on "User-UserGroup-Operation-Resource" to -flexibly control user roles and permissions (supports multiple GraphServers). +HugeGraph provides `StandardAuthenticator` for multi-user authentication and access control based on users, groups, operations, and resources. -Some key designs of the `StandardAuthenticator` mode include: -- During initialization, a super administrator (`admin`) user is created. Subsequently, other users can be created by the super administrator. Once newly created users are assigned sufficient permissions, they can create or manage more users. -- It supports dynamic creation of users, user groups, and resources, as well as dynamic allocation or revocation of permissions. -- Users can belong to one or multiple user groups. Each user group can have permissions to operate on any number of resources. The types of operations include read, write, delete, execute, and others. -- "Resource" describes the data in the graph database, such as vertices that meet certain criteria. Each resource consists of three elements: `type`, `label`, and `properties`. There are 18 types in total, with the ability to combine any label and properties. The internal condition of a resource is an AND relationship, while the condition between multiple resources is an OR relationship. +Initialization creates the `admin` superuser. Administrators create users, groups, and resources, then grant or revoke permissions. +Users receive read, write, delete, and execute permissions through their groups. -Here is an example to illustrate: +A resource describes accessible data using `type`, `label`, and `properties`. Conditions within one resource are combined with AND; multiple resources are +combined with OR. + +This relationship grants a user read access to data for Beijing. ```java // Scenario: A user only has data read permission for the Beijing area @@ -31,9 +29,12 @@ By default, HugeGraph does **not enable** user authentication, and it needs to b > [!WARNING] > **Production requires authentication** > -> HugeGraph disables user authentication by default. In production, enable authentication and authorization, set a strong non-default administrator password, maintain the Server IP allowlist, and grant minimum permissions. Do not expose Gremlin, Cypher, or other query endpoints directly to the public network. Standard Server configuration writes authentication-proxy audit records to `audit-*.log`; retain these files and restrict read access. `auth.audit_log_rate` limits per-user output rather than serving as a dedicated audit-log on/off switch. +> HugeGraph disables user authentication by default. In production, enable authentication and authorization, set a strong non-default administrator password, +maintain the Server IP allowlist, and grant minimum permissions. Do not expose Gremlin, Cypher, or other query endpoints directly to the public network. +Standard Server configuration writes authentication-proxy audit records to `audit-*.log`; retain these files and restrict read access. `auth.audit_log_rate` +limits per-user output rather than serving as a dedicated audit-log on/off switch. -You need to modify the configuration file to enable this feature. HugeGraph provides built-in authentication mode: `StandardAuthenticator`. This mode supports multi-user authentication and fine-grained permission control. Additionally, developers can implement their own `HugeAuthenticator` interface to integrate with their existing authentication systems. +Use `StandardAuthenticator` for built-in access control, or implement `HugeAuthenticator` to connect an existing identity system. HugeGraph uses [HTTP Basic Authentication](https://en.wikipedia.org/wiki/Basic_access_authentication). The value after `Basic` is the Base64 encoding of `username:password`. With curl, pass the credentials directly through `-u`: @@ -46,17 +47,23 @@ curl -u 'admin:' \ > [!WARNING] > **Protect Basic Authentication credentials** > -> Basic Authentication only Base64-encodes `username:password`; it does not encrypt credentials. Use HTTPS for network transmission. See [HTTPS configuration](config-https). +> Basic Authentication only Base64-encodes `username:password`; it does not encrypt credentials. Use HTTPS for network transmission. See [HTTPS +configuration](config-https). > [!WARNING] > **JWT risks in older versions** > -> HugeGraph-Server versions before 1.5.0 have JWT-related security risks in Auth mode. Upgrade affected versions or change the JWT `secretKey` according to that version's requirements. +> HugeGraph-Server versions before 1.5.0 have JWT-related security risks in Auth mode. Upgrade affected versions or change the JWT `secretKey` according to that +version's requirements. -`auth.token_secret` is read from the authentication graph's configuration. The default graph is `hugegraph`, so its file is normally `conf/graphs/hugegraph.properties`; if `auth.graph_store` changes, use that graph's properties file. -The source-code default generates 32 random bytes encoded as Base64, but does not write the value back to the file. Independently generated defaults cannot remain consistent across restarts or Server nodes. To preserve existing tokens across restarts or validate the same token on multiple nodes, explicitly configure the same strong random secret in each authentication graph configuration. +`auth.token_secret` is read from the authentication graph's configuration. The default graph is `hugegraph`, so its file is normally +`conf/graphs/hugegraph.properties`; if `auth.graph_store` changes, use that graph's properties file. +The core configuration defaults to 32 random bytes encoded as Base64 and does not write that default back to the file. Independently generated defaults cannot +remain consistent across restarts or Server nodes. To preserve existing tokens across restarts or validate the same token on multiple nodes, explicitly +configure the same strong random secret in each authentication graph configuration. -Server encodes this string as UTF-8 for JJWT's HS256 signer, which requires at least 32 bytes. This is a byte-length requirement; an arbitrary 32-character string need not have sufficient randomness. The command below generates 32 random bytes and Base64-encodes them locally. +Server encodes this string as UTF-8 for JJWT's HS256 signer, which requires at least 32 bytes. This is a byte-length requirement; an arbitrary 32-character +string need not have sufficient randomness. The command below generates 32 random bytes and Base64-encodes them locally. ```properties # Set the same value in every Server node's authentication graph configuration @@ -69,16 +76,16 @@ Generate the secret locally and transfer it securely into the authentication gra openssl rand -base64 32 ``` -Since the default is generated at every startup, explicitly configure it when tokens must survive restarts or be accepted by multiple nodes. Token lifetime is controlled by `auth.token_expire`, defaulting to 86400 seconds. +Token lifetime is controlled by `auth.token_expire`, defaulting to 86400 seconds. -Both `auth.token_expire` and `auth.token_secret` belong to the authentication graph specified by `auth.graph_store`. Entries with the same names in `rest-server.properties` do not override that graph configuration. +Both `auth.token_expire` and `auth.token_secret` belong to the authentication graph specified by `auth.graph_store`. Entries with the same names in +`rest-server.properties` do not override that graph configuration. #### StandardAuthenticator Mode -The `StandardAuthenticator` mode supports user authentication and permission control by storing user information in the database backend. This -implementation authenticates users based on their names and passwords (encrypted) stored in the database and controls user permissions based on their -roles. Below is the specific configuration process (requires service restart): +`StandardAuthenticator` stores users and password hashes in the backend and checks access by role. +After updating these three configurations, [initialize authentication and restart Server](#switching-authentication-mode). -Configure the `authenticator` and its `rest-server` file path in the `gremlin-server.yaml` configuration file: +Set the authenticator and REST configuration path in `conf/gremlin-server.yaml`. ```yaml authentication: { @@ -88,7 +95,7 @@ authentication: { } ``` -Configure the authenticator and the graph that stores authorization data in `rest-server.properties`: +Set the authenticator and authentication graph in `conf/rest-server.properties`. ```properties auth.authenticator=org.apache.hugegraph.auth.StandardAuthenticator @@ -102,9 +109,9 @@ auth.graph_store=hugegraph # auth.remote_url=127.0.0.1:8899,127.0.0.1:8898,127.0.0.1:8897 ``` -In the above configuration, the `graph_store` option specifies which graph to use for storing user information. If there are multiple graphs, you can choose any of them. +`auth.graph_store` names the graph that stores authentication data. It must match a graph configuration in `conf/graphs/`. -In the `hugegraph{n}.properties` configuration file, configure the `gremlin.graph` information: +Set `gremlin.graph` in `conf/graphs/.properties` for each protected graph. ```properties gremlin.graph=org.apache.hugegraph.auth.HugeFactoryAuthProxy @@ -114,66 +121,158 @@ For authorization API usage, see the [Authentication API](/docs/clients/restful- ### Custom User Authentication System -If you need to support a more flexible user system, you can customize the authenticator for extension. -Simply implement the `org.apache.hugegraph.auth.HugeAuthenticator` interface with your custom authenticator, -and then modify the `authenticator` configuration item in the configuration file to point to your implementation. +Implement `org.apache.hugegraph.auth.HugeAuthenticator` and point both `authenticator` configurations to that implementation to connect an existing identity +system. ### Switching authentication mode -When `init-store.sh` is run for the first time and the `admin` user does not yet exist, the command prompts for the -administrator password. For an initialized persistent backend, `init-store.sh` adds the system metadata required for -authentication without deleting existing graph data. +Use this sequence for a local persistent backend such as RocksDB, after applying the configuration above. +HStore/PD deployments create the administrator through the PD startup path. Set `auth.admin_pa` for that deployment instead of using local interactive +initialization. -```bash -# stop the hugeGraph firstly -bin/stop-hugegraph.sh +1. **Stop Server.** -# Initialize authentication system metadata; existing backend data is preserved -bin/init-store.sh + ```bash + bin/stop-hugegraph.sh + ``` -# start hugeGraph again -bin/start-hugegraph.sh +1. **Initialize authentication data.** -``` + ```bash + bin/init-store.sh + ``` + + Enter a strong password when prompted to create `admin`. Existing graph data is retained. If `admin` already exists, this command does not reset its + password. + Keep the existing storage directory. -### Use docker to enable authentication mode +1. **Start Server and check authentication.** -For versions of the hugegraph/hugegraph image equal to or greater than 1.2.0, you can enable authentication mode while starting the Docker image. + ```bash + bin/start-hugegraph.sh + curl -u admin \ + http://localhost:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels + ``` -The steps are as follows: + curl prompts for the password. Correct credentials should return HTTP 200; missing or incorrect credentials should return HTTP 401. + If startup fails, check the logs and authentication graph configuration before retrying. +{.steps} -#### 1. Use docker run +### Enable authentication with Docker -To enable authentication mode, add the environment variable `PASSWORD=xxx` (you can freely set the password) in the `docker run` command: +For a new container, choose either Docker run or Docker Compose. For an existing container, see +[enabling authentication in existing containers](#3-enter-the-container-to-enable-authentication-mode). + +When deploying with Docker, the official image supports enabling authentication +directly via environment variables (applicable to the official `/hugegraph-server` layout). +Replace the placeholder below with your image tag or digest: ```bash -docker run -itd -e PASSWORD=xxx --name=server -p 8080:8080 hugegraph/hugegraph:1.7.0 +export HUGEGRAPH_IMAGE='' +read -r -s -p 'Admin password: ' PASSWORD +printf '\n' +export PASSWORD ``` -#### 2. Use docker-compose +Execute the command and enter a strong administrator password when prompted. Avoid committing plaintext passwords into shared Compose files. +Persist the JWT secret in the authentication graph configuration as described [above](#configure-user-authentication). Use the same value across all nodes. + +#### Docker run {#1-use-docker-run} -Use `docker-compose` and set the environment variable `PASSWORD=xxx`: +Create a new container and publish its REST port only on the local host. + +```bash +docker run -d --name server \ + -p 127.0.0.1:8080:8080 \ + -e PASSWORD \ + "$HUGEGRAPH_IMAGE" +unset PASSWORD +``` + +The entrypoint configures authentication, initializes the backend on first startup, and starts Server. +Continue with [checking authentication](#check-authentication). + +#### Docker Compose {#2-use-docker-compose} + +Save the following as `compose.yaml`. Compose rejects missing required variables to avoid starting an unauthenticated service by accident. +Start the service in the same terminal. ```yaml -version: '3' services: server: - image: hugegraph/hugegraph:1.7.0 + image: ${HUGEGRAPH_IMAGE:?Set HUGEGRAPH_IMAGE} container_name: server ports: - - 8080:8080 + - "127.0.0.1:8080:8080" environment: - - PASSWORD=xxx + PASSWORD: ${PASSWORD:?Set PASSWORD} +``` + +```bash +docker compose up -d +unset PASSWORD ``` -#### 3. Enter the container to enable authentication mode +Set these variables again before recreating the container. Retain its data volumes and persist the configuration required by your deployment. +Continue with [checking authentication](#check-authentication). + +#### Existing containers {#3-enter-the-container-to-enable-authentication-mode} + +Use this path for an official Server container that ran without authentication and uses a local persistent backend. +Back up its data first. Confirm that the configuration, backend data, and `conf-bak` are all in mounted persistent volumes. +The official image declares `/hugegraph-server` as a volume; inspect the mounts of custom images first. +HStore/PD and external authentication services use their own administrator initialization paths. + +1. **Stop the original container and open a maintenance container with the same image and volumes.** + + ```bash + SERVER_IMAGE=$(docker inspect --format '{{.Image}}' server) + docker stop server + docker run --rm -it --volumes-from server \ + --entrypoint bash "$SERVER_IMAGE" + ``` + + Keep the original container stopped so two JVMs do not open the local database at once. `--entrypoint bash` opens a shell without starting Server. + +1. **Configure authentication and initialize it inside the maintenance container.** + + ```bash + cd /hugegraph-server + bin/enable-auth.sh + ``` + + On its first run, `enable-auth.sh` saves the original configuration in `conf-bak` and updates authentication and the default `hugegraph` graph. + If `conf-bak` already exists, the script skips its changes. Check all three configurations under [StandardAuthenticator mode](#standardauthenticator-mode). + Configure other authentication graphs manually. + Write a strong random JWT secret to the authentication graph configuration as described [above](#configure-user-authentication), then initialize it. + + ```bash + bin/init-store.sh + exit + ``` + + Enter the administrator password when prompted, then exit after initialization succeeds. + +1. **Start the original container from the host.** + + ```bash + docker start server + ``` + + Retain the `docker/init_complete` marker. Later changes to the `PASSWORD` environment variable do not change the stored administrator password. +{.steps} + +#### Check authentication -Enter the container first: +Once Server is running, execute the following requests from the host to verify. `-u admin` prompts interactively for the password: ```bash -docker exec -it server bash -# Modify the config quickly, the modified file are save in the conf-bak folder -bin/enable-auth.sh +curl -sS -o /dev/null -w '%{http_code}\n' \ + http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels +curl -sS -o /dev/null -w '%{http_code}\n' -u admin \ + http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/schema/vertexlabels ``` -Then follow [Switching authentication mode](#switching-authentication-mode) +Requests without credentials should return `401 Unauthorized`, while requests with the correct password return `200 OK` (incorrect passwords also return `401`). +Note: Current master URLs include `/graphspaces/DEFAULT`; for 1.7.0 and older versions, omit the graphspace prefix (use `/graphs/hugegraph/...` directly). +If unauthenticated requests still return `200`, authentication is not active; inspect configuration and restart Server. diff --git a/content/en/docs/guides/backup-restore.md b/content/en/docs/guides/backup-restore.md index 5f9f1f7769..9ab4e55723 100644 --- a/content/en/docs/guides/backup-restore.md +++ b/content/en/docs/guides/backup-restore.md @@ -6,102 +6,129 @@ weight: 5 ## Description -Backup and Restore are functions of backup map and restore map. The data backed up and restored includes metadata (schema) and graph data (vertex and edge). +Backup exports a graph's schema, vertices, and edges as JSON data. Restore imports that data into a target graph. +These [HugeGraph-Tools](/docs/quickstart/toolchain/hugegraph-tools) commands support migration, recovery after an upgrade, and graph merging. #### Backup -Export the metadata and graph data of a graph in the HugeGraph system in JSON format. +Backup runs in any graph mode. Pausing source writes before exporting is recommended to keep schema, vertices, and edges consistent +(paginated read APIs do not provide a global transactional snapshot). +Note: `backup -t all` only exports schema and graph data, excluding deployment configurations and user permissions. #### Restore -Re-import the data in JSON format exported by Backup to a graph in the HugeGraph system. +Before restoring data, switch the target graph to the appropriate mode. +Normal read/write operations use `NONE` mode; note that `LOADING` mode cannot be used for `restore`. -Restore has two modes: +| Target | Mode | Schema IDs | Vertex and edge IDs | +|---|---|---|---| +| New graph without schema or data | `RESTORING` | Use IDs from the backup | Use IDs from the backup | +| Existing graph to merge into | `MERGING` | Use target schema IDs, creating schema if needed | Some IDs change with target schema; see below | -- In Restoring mode, the metadata and graph data exported by Backup are restored to the HugeGraph system intact. It can be used for graph backup and recovery, and the general target graph is a new graph (without metadata and graph data). for example: - - System upgrade, first back up the map, then upgrade the system, and finally restore the map to the new system - - Graph migration, from a HugeGraph system, use the Backup function to export the graph, and then use the Restore function to import the graph into another HugeGraph system -- In the Merging mode, the metadata and graph data exported by Backup are imported into another graph that already has metadata or graph data. During the process, the ID of the metadata may change, and the IDs of vertices and edges will also change accordingly. - - Can be used to merge graphs +In `MERGING`, Tools recomputes `PRIMARY_KEY` vertex IDs using the target label IDs and updates connected edge endpoints and IDs. +It retains `AUTOMATIC` and custom vertex IDs. The merge reuses schema with matching names. +Check that their definitions are compatible and that retained vertex IDs do not conflict. +It does not resolve arbitrary schema or data conflicts. Test on a separate target before merging production data. ## Instructions -You can use [hugegraph-tools](/docs/quickstart/toolchain/hugegraph-tools) to backup and restore the graph. +Prepare [HugeGraph-Tools](/docs/quickstart/toolchain/hugegraph-tools) compatible with your Server version, then run the commands from the Tools root directory. +This example restores a source graph on port `8080` to an empty target on port `8081`. +Both Servers and the target graph must already exist and be running. Adjust the addresses and graph names for your deployment. + +```bash +SOURCE_URL=http://127.0.0.1:8080 +TARGET_URL=http://127.0.0.1:8081 +SOURCE_GRAPH=hugegraph +TARGET_GRAPH=hugegraph +BACKUP_DIR="$PWD/data/hugegraph-backup" +``` + +With authentication enabled, add `--user --password ` before the subcommand in each Tools command. +The account needs the relevant graph access and graph-mode management permissions. +See the [Tools reference](/docs/quickstart/toolchain/hugegraph-tools) for connection and credential options. #### Backup +Pause source writes, then export all schema, vertices, and edges. + ```bash -bin/hugegraph backup -t all -d data +bin/hugegraph --url "$SOURCE_URL" --graph "$SOURCE_GRAPH" \ + backup -t all -d "$BACKUP_DIR" ``` -This command backs up all the metadata and graph data of the hugegraph graph of http://127.0.0.1:8080 (the default --url) to the data directory. - -> Backup works in any graph mode, it does not check the graph mode +An explicit `-d` is the backup directory itself; the graph name is not appended. Use a separate directory for each backup to avoid mixing old files. +Retain the complete directory, including compressed files. Restore reads the JSON backup formats supported by Tools. #### Restore -Restore has two modes: RESTORING and MERGING. Before restore, you must first set the graph mode according to your needs, the restore command fails when the graph is in any other mode. +This sequence uses `RESTORING`. To merge, use `MERGING` after the compatibility checks above. +Pause application writes to the target during maintenance. -##### Step 1: View and set graph mode -```bash -bin/hugegraph graph-mode-get -``` -This command is used to view the current graph mode, including: NONE, RESTORING, MERGING, LOADING. +1. **Record the original mode, then switch it.** -```bash -bin/hugegraph graph-mode-set -m RESTORING -``` + ```bash + bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" graph-mode-get + ORIGINAL_MODE=NONE + bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" \ + graph-mode-set -m RESTORING + ``` -This command is used to set the graph mode. Before Restore, it can be set to RESTORING or MERGING mode. In the example, it is set to RESTORING. + A graph in normal use should be in `NONE` mode. Set `ORIGINAL_MODE` to the value returned by the query, such as `NONE` below. + If another task is using `RESTORING`, `MERGING`, or `LOADING`, coordinate with that task before changing its mode. -##### Step 2: Restore data +1. **Restore the data.** -```bash -bin/hugegraph restore -t all -d data -``` -This command re-imports all metadata and graph data in the data directory to the hugegraph graph at http://127.0.0.1:8080. + ```bash + bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" \ + restore -t all -d "$BACKUP_DIR" + ``` -##### Step 3: Restoring Graph Mode + Check command output and the restore results for each type. On failure, retain the backup directory and logs to inspect target state. + Restore does not automatically roll back committed writes; do not re-run on a partially restored graph without inspection. -```bash -bin/hugegraph graph-mode-set -m NONE -``` -This command is used to restore the graph mode to NONE. +1. **Restore the original mode.** -So far, a complete graph backup and graph recovery process is over. + Reset the mode using `ORIGINAL_MODE`, recorded in the first step. -#### help + ```bash + bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" graph-mode-set -m "$ORIGINAL_MODE" + bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" graph-mode-get + ``` -For detailed usage of backup and restore commands, please refer to the [hugegraph-tools documentation](/docs/quickstart/toolchain/hugegraph-tools). + Regardless of whether restore succeeds or fails, always restore the graph mode back to its original state (usually `NONE`). + On failure, preserve the logs and state, and re-prepare the target graph before resuming business writes. +{.steps} -## API description for Backup/Restore usage and implementation +After resetting the mode, compare vertex and edge counts on both graphs. -#### Backup +```bash +bin/hugegraph --url "$SOURCE_URL" --graph "$SOURCE_GRAPH" gremlin-execute --script 'g.V().count()' +bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" gremlin-execute --script 'g.V().count()' +bin/hugegraph --url "$SOURCE_URL" --graph "$SOURCE_GRAPH" gremlin-execute --script 'g.E().count()' +bin/hugegraph --url "$TARGET_URL" --graph "$TARGET_GRAPH" gremlin-execute --script 'g.E().count()' +``` -Backup uses the corresponding list(GET) API export of metadata and graph data, and no new API is added. +Each pair of counts should match exactly. We also recommend using the [Schema API](/docs/clients/restful-api/schema/) to verify metadata definitions, +and sampling vertex and edge properties and endpoints to ensure complete data integrity. -#### Restore +#### Help -Restore uses the corresponding create(POST) API imports for metadata and graph data, and does not add new APIs. +For additional options on compression, concurrency, and selective type backup, see the [Tools reference](/docs/quickstart/toolchain/hugegraph-tools). +Before performing cross-major-version restores, verify backup file compatibility in a test environment first. -There are two different modes for Restore: Restoring and Merging. In addition, there is a regular mode of NONE (default), the differences are as follows: +## API description for Backup/Restore usage and implementation -- In None mode, the writing of metadata and graph data is normal, please refer to the function description. special: - - ID is not allowed when metadata (schema) is created - - Graph data (vertex) is not allowed to specify an ID when the id strategy is Automatic -- Restoring mode, restoring to a new graph, in particular: - - ID is allowed to be specified when metadata (schema) is created - - Graph data (vertex) allows specifying an ID when the id strategy is Automatic -- Merging mode, merging into a graph with existing metadata and graph data, in particular: - - ID is not allowed when metadata (schema) is created - - Graph data (vertex) allows specifying an ID when the id strategy is Automatic +#### Backup +Backup exports data through the schema, vertex, and edge list GET APIs. -Normally, the graph mode is None. When you need to restore the graph, you need to temporarily change the graph mode to Restoring mode or -Merging mode as needed, and when the Restore is completed, restore the graph mode to None. +#### Restore -The implemented RESTful API for setting graph mode is as follows: +Restore imports data through the schema, vertex, and edge creation APIs. See the [mode table](#restore) for ID behavior. +`RESTORING` accepts schema IDs; `MERGING` does not. Both accept retained `AUTOMATIC` vertex IDs. +Normal `NONE` mode does not accept these ID settings required for restore. ##### View the mode of a graph @@ -127,7 +154,7 @@ GET http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/mode } ``` -> Legal graph modes include: NONE, RESTORING, MERGING, LOADING +Valid graph modes are `NONE`, `RESTORING`, `MERGING`, and `LOADING`. ##### Set the mode of a graph @@ -145,7 +172,7 @@ PUT http://localhost:8080/graphspaces/DEFAULT/graphs/{graph}/mode "RESTORING" ``` -> Legal graph modes include: NONE, RESTORING, MERGING, LOADING +Valid graph modes are `NONE`, `RESTORING`, `MERGING`, and `LOADING`. ###### Response Status diff --git a/content/en/docs/guides/faq.md b/content/en/docs/guides/faq.md index ff5aa206e6..b6fc1c4fd7 100644 --- a/content/en/docs/guides/faq.md +++ b/content/en/docs/guides/faq.md @@ -4,100 +4,157 @@ linkTitle: "FAQ" weight: 6 --- -- How to choose the back-end storage? RocksDB or distributed storage? +## Deployment and startup {#deployment} - HugeGraph supports multiple deployment modes. Choose based on your data scale and scenario: - - **Standalone Mode**: Server + RocksDB, suitable for development, testing, Graph AI, and small to medium-scale production environments (≤ 2 TB) - - **Distributed Mode**: HugeGraph-PD + HugeGraph-Store (HStore), for deployments that require horizontal scaling and multiple replicas, supporting data scales up to 1 PB +### Choose RocksDB or HStore {#choose-backend} - Version 1.7.0 supports RocksDB, HStore, HBase, and Memory. Legacy backends such as Cassandra, ScyllaDB, MySQL, and PostgreSQL require version 1.5.x or earlier. +For development, testing, or small-to-medium standalone deployments, **Server + RocksDB** is recommended for its simplicity and high +performance. When data volume is large (TB to PB scale) or requires horizontal scalability and multi-replica high availability, deploy an +**HStore distributed cluster** (PD + Store + Server). -- Prompt when starting the service: `xxx (core dumped) xxx` - - First confirm that your JDK version matches the requirements: Java 17 for HugeGraph 1.8.0+, or Java 11 for 1.7.0. HugeGraph no longer supports Java 8. +Version 1.7.0 supports RocksDB, HStore, HBase, and Memory. Legacy backends such as Cassandra, ScyllaDB, MySQL, and PostgreSQL require a +version earlier than 1.7.0. See the [Server quick start](/docs/quickstart/hugegraph/hugegraph-server/) for deployment steps. -- The service is started successfully, but there is a prompt similar to "Unable to connect to the backend or the connection is not open" when operating the graph +### Investigate a core dump at startup {#core-dump} - Persistent local backends such as RocksDB and HBase must be initialized with `init-store` before their first startup. HStore is managed by PD and Store and does not use this script. - -- Do all backends need to be executed before use init-store, and can the serialization options be filled in at will? +First, run `java -version` to confirm your JDK version: **HugeGraph master (1.8 baseline) requires Java 17**, and **1.7.0 requires Java 11** (Java 8 is no longer supported). If the JDK version is correct, inspect the Server logs and the JVM crash log (`hs_err_pid*.log`) to check for +architecture mismatches, missing native libraries (such as RocksDB JNI), or insufficient system memory. - Memory and HStore do not use `init-store`; persistent local backends such as RocksDB and HBase must be initialized before first use. The serializer must match the backend, for example RocksDB uses `binary`. +### Which backends need init-store {#initialize-backend} -- Execution `init-store` error: ```Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp/librocksdbjni3226083071221514754.so: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.10' not found (required by /tmp/librocksdbjni3226083071221514754.so)``` +When using local persistent backends like RocksDB or HBase, **you must run `bin/init-store.sh` before the first startup**. Memory (in-memory +backend) and HStore (distributed backend) do not use this script; HStore storage is managed directly by PD and Store. - RocksDB requires gcc 4.3.0 (GLIBCXX_3.4.10) and above +Note: The `backend` and `serializer` settings must match (for example, RocksDB requires `binary`). If the service starts but reports that +the backend is disconnected, verify initialization and backend connection settings first. -- The `bin` directory contains `start-hugegraph.sh`, `start-restserver.sh` and `start-gremlinserver.sh`. These scripts seem to be related to startup. Which one should be used? +### Resolve a missing GLIBCXX version during init-store {#native-library} - Current release packages retain only `start-hugegraph.sh` as the Server startup script. GremlinServer and the REST Server run in the same process. +The following error indicates that the host system's `libstdc++` version is too old and lacks symbols required by the RocksDB JNI native library: -- Two graphs are configured, the names are `hugegraph` and `hugegraph1`, and the command to start the service is `start-hugegraph.sh`. Is only the hugegraph graph opened? +```text +java.lang.UnsatisfiedLinkError: .../librocksdbjni....so: +.../libstdc++.so.6: version `GLIBCXX_3.4.10' not found +``` - The script name is unrelated to the graph name. Server scans and loads local graph configurations from `conf/graphs` by default; change the directory with - `graphs` in `conf/rest-server.properties`. `graph.load_from_local_config` defaults to `false` and controls constructor preloading, not whether startup loads - local graphs. Give each local graph its own configuration file and restart Server after changing it. See the - [Server configuration guide](/docs/config/config-guide/). +To resolve: Upgrade the system's `libstdc++` (or install a newer GCC runtime), or run HugeGraph using the official Docker image. -- After the service starts successfully, garbled characters are returned when using `curl` to query all vertices +### Which Server startup script should I use {#start-script} - The batch vertices/edges returned by the server are compressed (gzip), and can be redirected to `gunzip` for decompression (`curl http://example | gunzip`), or can be sent with the `postman` of `Firefox` or the `restlet` plug-in of Chrome browser. request, the response data will be decompressed automatically. - -- When using the vertex Id to query the vertex through the `RESTful API`, it returns empty, but the vertex does exist +Current releases uniformly use `bin/start-hugegraph.sh`. Both GremlinServer and the REST Server start in the same process. The +`start-restserver.sh` and `start-gremlinserver.sh` scripts from older documentation are deprecated. - Check the type of the vertex ID. If it is a string type, the "id" part of the API URL needs to be enclosed in double quotes, while for numeric types, it is not necessary to enclose the ID in quotes. +### Which graphs does the startup script load {#load-graphs} -- Vertex Id has been double quoted as required, but querying the vertex via the RESTful API still returns empty - - Check whether the vertex id contains `+`, `space`, `/`, `?`, `%`, `&`, and `=` reserved characters of these `URLs`. If they exist, they need to be encoded. The following table gives the coded values: - - ```text - special character | encoded value - ------------------| ------------- - + | %2B - space | %20 - / | %2F - ? | %3F - % | %25 - # | %23 - & | %26 - = | %3D - ``` - -- Timeout when querying vertices or edges of a certain category (`query by label`) +The script name has no relation to the graph name. Server scans and loads all graph configurations in `conf/graphs` by default (this +directory can be customized via `graphs` in `conf/rest-server.properties`). Each local graph uses an independent configuration file; restart +Server after editing configurations. - Since the amount of data belonging to a certain label may be relatively large, please add a limit limit. +Note: `graph.load_from_local_config` defaults to `false` (controlling constructor preloading) and cannot prevent local graph loading at +startup. See the [Server configuration guide](/docs/config/config-guide/). -- It is possible to operate the graph through the `RESTful API`, but when sending `Gremlin` statements, an error is reported: `Request Failed(500)` +## Connections and queries {#queries} - It may be that the configuration of `GremlinServer` is wrong, check whether the `host` and `port` of `gremlin-server.yaml` match the `gremlinserver.url` of `rest-server.properties`, if they do not match, modify them, and then Restart the service. +### Read compressed responses with curl {#compressed-response} -- When using `Loader` to import data, a `Socket Timeout` exception occurs, and then `Loader` is interrupted +Batch vertex and edge endpoints return gzip-compressed data by default. Add the `--compressed` flag to curl to decompress automatically: - Continuously importing data will put too much pressure on the `Server`, which will cause some requests to time out. The pressure on `Server` can be appropriately relieved by adjusting the parameters of `Loader` (such as: number of retries, retry interval, error tolerance, etc.), and reduce the frequency of this problem. +```bash +curl --compressed \ + 'http://127.0.0.1:8080/graphspaces/DEFAULT/graphs/hugegraph/graph/vertices?limit=10' +``` -- How to delete all data from a graph +If authentication is enabled, supply credentials. If output remains unexpected, check HTTP status codes, `Content-Type`, and +`Content-Encoding`. Do not pipe JSON error responses directly into `gunzip`. - An administrator can call `DELETE /graphspaces/{graphspace}/graphs/{graph}/clear?confirm_message=I'm sure to delete all data`. The `confirm_message` query parameter must match that value exactly, otherwise the request is rejected. See the [Graph API](../clients/restful-api/graphs) for details. This operation removes schemas, vertices, edges, and indexes. +### A vertex exists, but lookup by ID cannot find it {#vertex-id} -- The database has been cleared and `init-store` has been executed, but when trying to add a schema, the prompt "xxx has existed" appeared. +Check the exact format and type of `id` in existing query results: **string IDs require JSON double quotes, while numeric IDs do not**. For +example, the string ID `1:marko` is written in JSON as `"1:marko"`. - There is a cache in the `HugeGraphServer`, and it is necessary to restart the `Server` when the database is cleared, otherwise the residual cache will be inconsistent. +When embedding this JSON ID into a URL path, URL-encode the entire string including double quotes. If the ID contains special characters, +encoding the whole quoted string is required: -- An error is reported during the process of inserting vertices or edges: `The max length of vertex id is 16384, but got xxx {yyy}` or `The max length of edge id is 65536, but got xxx {yyy}` +| Character | URL encoding | +| --- | --- | +| `+` | `%2B` | +| Space | `%20` | +| `/` | `%2F` | +| `?` | `%3F` | +| `%` | `%25` | +| `#` | `%23` | +| `&` | `%26` | +| `=` | `%3D` | +| `"` | `%22` | - In order to ensure query performance, the current backend storage limits the length of the id column. The vertex id cannot exceed 16384 bytes and the edge id cannot exceed 65536 bytes. An index id longer than 32 bytes is stored as a hash instead of being rejected. +### Queries by label time out {#label-timeout} -- Is there support for nested attributes, and if not, are there any alternatives? +When a label contains a large volume of data, scanning all records will exhaust resources and time out. Recommended solutions: +1. Append `.limit(n)` to restrict the number of returned records; +2. Use pagination to fetch results incrementally; +3. Filter by specific properties and ensure the relevant properties are indexed to prevent full scans. - Nested attributes are currently not supported. Alternative: Nested attributes can be taken out as individual vertices and connected with edges. +### REST works, but Gremlin returns HTTP 500 {#gremlin-connection} -- Can an `EdgeLabel` connect multiple pairs of `VertexLabel`, such as "investment" relationship, which can be "individual" investing in "enterprise", or "enterprise" investing in "enterprise"? +A working REST endpoint only confirms HTTP reachability. HTTP 500 from Gremlin typically stems from: +1. **Gremlin Server connection failure**: Verify that `gremlinserver.url` in `conf/rest-server.properties` points to a reachable Gremlin +port (default `8182`, configured in `conf/gremlin-server.yaml`); +2. **Query syntax or logic error**: Such as unindexed property filtering triggering scan limits; +3. **Missing or expired credentials**. - Yes. Call `link(sourceLabel, targetLabel)` once per pair when building the `EdgeLabel`; every pair is kept, so one "investment" label can cover both "individual" to "enterprise" and "enterprise" to "enterprise". The older `sourceLabel()` and `targetLabel()` builder methods are deprecated and accept only a single pair. +Check `logs/hugegraph-server.log` for the detailed exception stack trace. -- Prompt `HTTP 415 Unsupported Media Type` when sending a request through `RestAPI` +### REST returns HTTP 415 {#content-type} - `Content-Type: application/json` needs to be specified in the request header +When sending POST/PUT requests with a JSON body, set the header `Content-Type: application/json` and ensure the payload is valid JSON. -Other issues can be searched in the issue area of the corresponding project, such as [Server-Issues](https://github.com/apache/hugegraph/issues) / [Loader Issues](https://github.com/apache/hugegraph-loader/issues) +## Import and data maintenance {#data-management} + +### Loader reports Socket Timeout {#loader-timeout} + +Import timeouts usually occur when batch sizes are too large or write concurrency is too high for the Server. Recommended actions: +1. Decrease `--batch-size` (e.g., from 500 down to 200); +2. Reduce the number of concurrent insert threads; +3. Increase `--timeout` and configure retry settings. See the [Loader guide](/docs/quickstart/toolchain/hugegraph-loader/) for parameter details. + +### Clear all data from a graph {#clear-graph} + +Administrators can call the following endpoint to wipe schema, vertices, edges, and indexes from a graph (verify target graph and backups beforehand): + +```text +DELETE /graphspaces/{graphspace}/graphs/{graph}/clear +``` + +The request requires the `confirm_message` query parameter, strictly matching `I'm sure to delete all data` (URL-encoded). See the [Graph +API](/docs/clients/restful-api/graphs/) for complete reference. + +### Schema still exists after clearing the backend {#schema-cache} + +Server caches metadata in memory. Bypassing Server to clear underlying storage directly (such as wiping RocksDB directories or HBase tables) +causes cache inconsistency. If storage was manually deleted, **restart Server** to clear the cache. For normal cleanup, use the official +[Clear Graph API](/docs/clients/restful-api/graphs/#clear-graph). + +## Modeling and IDs {#modeling} + +### A vertex or edge ID exceeds the length limit {#id-length} + +To maintain storage and indexing performance, HugeGraph enforces maximum ID length limits (calculated by encoded byte count): +- **Vertex ID**: Maximum 16384 bytes (16 KB); +- **Edge ID**: Maximum 65536 bytes (64 KB). + +Use concise business identifiers as primary keys rather than concatenating large text blocks. Note: Property index values exceeding 32 bytes +are automatically hashed for storage and will not trigger length errors. + +### Are nested properties supported {#nested-properties} + +Nested properties (complex nested maps or objects) are not currently supported. To represent multi-level relations, model the nested object +as an independent vertex and link it to the parent vertex with an edge. + +### Can one edge label connect several vertex-label pairs {#edge-links} + +Yes. When creating an `EdgeLabel`, invoke `link(sourceLabel, targetLabel)` once for each source-target pair. +For example, an `investment` edge label can link both `individual → enterprise` and `enterprise → enterprise`. The deprecated +`sourceLabel()` and `targetLabel()` methods only supported a single pair and should no longer be used. + +For other questions, search the [Server issues](https://github.com/apache/hugegraph/issues) or [Toolchain +issues](https://github.com/apache/hugegraph-toolchain/issues). diff --git a/content/en/docs/images/hubble/accounts.jpg b/content/en/docs/images/hubble/accounts.jpg index 8cc3af1a0b..8deb0e9206 100644 Binary files a/content/en/docs/images/hubble/accounts.jpg and b/content/en/docs/images/hubble/accounts.jpg differ diff --git a/content/en/docs/images/hubble/algorithms.jpg b/content/en/docs/images/hubble/algorithms.jpg index 52a6c4e0fc..bc6a26cad2 100644 Binary files a/content/en/docs/images/hubble/algorithms.jpg and b/content/en/docs/images/hubble/algorithms.jpg differ diff --git a/content/en/docs/images/hubble/cluster-overview.jpg b/content/en/docs/images/hubble/cluster-overview.jpg index e7d628f641..15a1b880c2 100644 Binary files a/content/en/docs/images/hubble/cluster-overview.jpg and b/content/en/docs/images/hubble/cluster-overview.jpg differ diff --git a/content/en/docs/images/hubble/graphspace.jpg b/content/en/docs/images/hubble/graphspace.jpg index 9eaa534673..656dc1b9b8 100644 Binary files a/content/en/docs/images/hubble/graphspace.jpg and b/content/en/docs/images/hubble/graphspace.jpg differ diff --git a/content/en/docs/images/hubble/members.jpg b/content/en/docs/images/hubble/members.jpg index 7d6a430278..dd453f37da 100644 Binary files a/content/en/docs/images/hubble/members.jpg and b/content/en/docs/images/hubble/members.jpg differ diff --git a/content/en/docs/images/hubble/overview.jpg b/content/en/docs/images/hubble/overview.jpg index 91c78b9ae7..57b51d8da7 100644 Binary files a/content/en/docs/images/hubble/overview.jpg and b/content/en/docs/images/hubble/overview.jpg differ diff --git a/content/en/docs/images/hubble/query.jpg b/content/en/docs/images/hubble/query.jpg index c0bcb0d454..79b6cbc5dc 100644 Binary files a/content/en/docs/images/hubble/query.jpg and b/content/en/docs/images/hubble/query.jpg differ diff --git a/content/en/docs/images/hubble/schema.jpg b/content/en/docs/images/hubble/schema.jpg index 3d4c1a463f..7beb949fcd 100644 Binary files a/content/en/docs/images/hubble/schema.jpg and b/content/en/docs/images/hubble/schema.jpg differ diff --git a/content/en/docs/images/hubble/store-node.jpg b/content/en/docs/images/hubble/store-node.jpg index 686da5a587..b5c5c8aa6c 100644 Binary files a/content/en/docs/images/hubble/store-node.jpg and b/content/en/docs/images/hubble/store-node.jpg differ diff --git a/content/en/docs/images/hubble/text2gql-prompt.jpg b/content/en/docs/images/hubble/text2gql-prompt.jpg index f147aeecf4..e163268931 100644 Binary files a/content/en/docs/images/hubble/text2gql-prompt.jpg and b/content/en/docs/images/hubble/text2gql-prompt.jpg differ diff --git a/content/en/docs/images/hubble/text2gql-settings.jpg b/content/en/docs/images/hubble/text2gql-settings.jpg index 1fdbef472f..d9ffc2c8d7 100644 Binary files a/content/en/docs/images/hubble/text2gql-settings.jpg and b/content/en/docs/images/hubble/text2gql-settings.jpg differ diff --git a/content/en/docs/images/hubble/text2gql.jpg b/content/en/docs/images/hubble/text2gql.jpg index 08fcebdd13..5a061ee3d4 100644 Binary files a/content/en/docs/images/hubble/text2gql.jpg and b/content/en/docs/images/hubble/text2gql.jpg differ diff --git a/content/en/docs/quickstart/client/hugegraph-client.md b/content/en/docs/quickstart/client/hugegraph-client.md index e299497105..e8dac27940 100644 --- a/content/en/docs/quickstart/client/hugegraph-client.md +++ b/content/en/docs/quickstart/client/hugegraph-client.md @@ -12,16 +12,17 @@ For other languages, use the [Go Client](/docs/quickstart/client/hugegraph-clien ### 2 What You Need -- JDK 11 (used by the current CI; the source target remains Java 8) -- Maven 3.6+ +- JDK 17 for this walkthrough and for building Client `1.8.0` from Toolchain `master` +- Maven 3.6.3+ ### 3 How To Use The basic steps to use HugeGraph-Client are as follows: -- Build a new Maven project by IDEA or Eclipse -- Add HugeGraph-Client dependency in a pom file; -- Create an object to invoke the interface of HugeGraph-Client +1. Create a Maven project in IntelliJ IDEA or Eclipse. +1. Add the HugeGraph-Client dependency to `pom.xml`. +1. Create a client, call its APIs, and close it after use. +{.steps} See the complete example in the following section for the detail. @@ -46,25 +47,41 @@ Using IDEA or Eclipse to create the project: ``` -The complete examples below use Schema, Graph, Gremlin and resource-cleanup APIs available in the published Client `1.7.0` -and compatible with Toolchain `master`. Use the dependency above directly in your example project. -For new master APIs in the [API reference](/docs/clients/hugegraph-client/), such as capability detection, use Client `1.8.0` -from Toolchain `master` and change the POM version to `1.8.0`. If that version is not published to Maven yet, run the following -command from the Toolchain source root to install the Client and its parent POM locally. The source build still needs -its dependencies available from your configured Maven repositories. +Choose the client version before building: + +| Use case | Client version | Java build target | +| --- | --- | --- | +| Run the complete examples below | Published `1.7.0`; use the dependency above | Java 8 in the release POM; this walkthrough uses JDK 17 | +| Use new master APIs, such as capability detection | Toolchain `master` (based on 1.8) | Java 17 | + +The examples use Schema, Graph, Gremlin and resource-cleanup APIs available in Client `1.7.0` and compatible with Toolchain `master`. +The release's Java build target does not describe the runtime requirements of every transitive dependency. + +To use unreleased features from the master branch, build Client from source (see [Tools source build instructions](/docs/quickstart/toolchain/hugegraph-tools/#build-current-source)): ```bash -mvn install -pl hugegraph-client -am -DskipTests -Dmaven.javadoc.skip=true -ntp +java -version +mvn -version +mvn -Dmaven.repo.local="${candidate_dir:?Run the candidate SDK setup first}/m2" \ + -Dsdk.validation.mode=candidate \ + install -pl hugegraph-client -am -DskipTests -Dmaven.javadoc.skip=true -ntp ``` -Client and Server versions need not match. Check the supported server API range and required capabilities before upgrading. +Both version checks must report JDK 17. Also pass `-Dmaven.repo.local="$candidate_dir/m2"` to Maven commands in projects using Client from master. +Client and Server versions need not be identical; check the required APIs and compatibility before upgrading. #### 4.3 Example +Choose a single-record or batch example. Save the complete code as `src/main/java/SingleExample.java` or +`src/main/java/BatchExample.java` in your Maven project. The copy button includes the folded lines. + +{{< tabs group="java-client-example" default="single" label="Java client example" >}} +{{< tab label="SingleExample" value="single" >}} + ##### 4.3.1 SingleExample -```java +```java {collapse=30} import java.io.IOException; import java.util.Iterator; import java.util.List; @@ -216,9 +233,12 @@ public class SingleExample { } ``` +{{< /tab >}} +{{< tab label="BatchExample" value="batch" >}} + ##### 4.3.2 BatchExample -```java +```java {collapse=30} import java.util.ArrayList; import java.util.List; @@ -359,9 +379,17 @@ public class BatchExample { } ``` +{{< /tab >}} +{{< /tabs >}} + #### 4.4 Run The Example -Before running Example, you need to start the Server. For the startup process, see[HugeGraph-Server Quick Start](/docs/quickstart/hugegraph/hugegraph-server). +1. [Start HugeGraph Server](/docs/quickstart/hugegraph/hugegraph-server) and prepare a test graph named `hugegraph` in graphspace `DEFAULT`. +1. Set the Maven project's JDK to 17. In the selected example, replace the Server URL, graphspace, graph and credentials in `HugeClient.builder(...)` + and `configUser(...)` with your settings. Omit `configUser(...)` if Server does not require authentication. +1. Run `SingleExample.main()` or `BatchExample.main()` in your IDE. Both examples create schema and graph data, print the results, and close the client. + Use a test graph: existing schema or data can affect repeated runs. +{.steps} #### 4.5 More Information About Client-API diff --git a/content/en/docs/quickstart/hugegraph-ai/quick_start.md b/content/en/docs/quickstart/hugegraph-ai/quick_start.md index 6267b32652..2405891fdf 100644 --- a/content/en/docs/quickstart/hugegraph-ai/quick_start.md +++ b/content/en/docs/quickstart/hugegraph-ai/quick_start.md @@ -4,12 +4,52 @@ linkTitle: "LLM Workflow" weight: 3 --- -This page explains the processing flow in the HugeGraph-LLM Web UI. See [HugeGraph-LLM](./hugegraph-llm.md) for startup instructions. +Follow the Web UI to build indexes, query documents, and generate Gremlin. See [HugeGraph-LLM](./hugegraph-llm.md) for startup instructions. + +Prepare a reachable HugeGraph Server, a generation model, and an embedding model. Viewing settings or running handwritten Gremlin in +`Graph Tools` needs no model. Document vectors require embeddings; graph extraction and answers also require an LLM. +The reranker is optional. Start with the local `bleu` method. + +| Task | UI tab | Prerequisites | +| --- | --- | --- | +| Build indexes from documents | `1. Build RAG Index` | Documents and embeddings; graph extraction also needs a schema and LLM | +| Answer from documents or a graph | `2. (Graph)RAG & User Functions` | The corresponding indexes and an LLM | +| Convert a question to Gremlin | `3. Text2gremlin` | Graph schema and LLM; examples also need embeddings and an example index | +| Check graph data directly | `4. Graph Tools` | A HugeGraph connection and handwritten Gremlin | + +The screenshots show the HugeGraph-LLM Web UI. The built-in prompt language can be toggled via `LANGUAGE` in `.env`, while UI controls remain in English. + +> [!DETAILS]- Missing batch-question template when starting from source +> +> If startup raises `FileNotFoundError` for `questions_template.xlsx`, create the input template from the repository root after installing dependencies, +> then restart. This preserves an existing file. The template has all 6 columns, with a test question and expected answer; the 4 model-answer columns stay empty. +> +> ```bash +> uv run --extra llm python - <<'PYTHON' +> from pathlib import Path +> from openpyxl import Workbook +> +> path = Path("hugegraph-llm/src/hugegraph_llm/resources/demo/questions_template.xlsx") +> if not path.exists(): +> workbook = Workbook() +> sheet = workbook.active +> sheet.append([ +> "Question", "Expected Answer", "Basic LLM Answer", +> "Vector-only Answer", "Graph-only Answer", "Graph-Vector Answer", +> ]) +> sheet.append(["Who is Sarah?", "Sarah is a 30-year-old attorney.", None, None, None, None]) +> workbook.save(path) +> PYTHON +> ``` ## 0. Configuration Panel Above the tabs sits a collapsible configuration panel with five sections: `1. Set up the HugeGraph server.`, `2. Set up the LLM.`, `3. Set up the Embedding.`, `4. Set up the Reranker.`, and `5. Set up the vector engine.`. Each section has its own apply button, and applying a change writes the supported fields back to `.env`. The header also shows the current prompt language. +Set the answer, extraction, and query-generation models in the `chat`, `mini_tasks`, and `text2gql` sub-tabs of the LLM settings. +The screenshots use local Ollama models `qwen2.5:3b` and `mxbai-embed-large`, with `Faiss` as the vector engine. +Rebuild indexes after changing embedding models, since vector dimensions or meaning can differ. + ## 1. Build RAG Indexes The first tab splits documents into a chunk vector index. It also extracts vertices and edges according to a schema, writes them to HugeGraph, and maintains a vertex vector index. @@ -26,7 +66,47 @@ flowchart TD Input comes from either the `text` sub-tab or the `file` sub-tab. Uploads accept `.txt`, `.docx`, and `.pdf`, and several files can be selected at once. -Common operations are `Import into Vector`, `Extract Graph Data (1)`, `Load into GraphDB (2)`, and `Update Vid Embedding`. `Load into GraphDB (2)` also refreshes the vertex vector index, so the separate `Update Vid Embedding` step is only needed when the graph already held data. The `Graph Extraction Split Type` dropdown next to these buttons chooses `document`, `paragraph`, or `sentence`. `document` keeps the whole input as one unit; the other two split long documents before extraction. +For a first run, paste this short document into `Input Doc(s)`. Select a dedicated test graph before importing it. + +```text +Sarah is a 30-year-old attorney. James is a journalist. Sarah has been James's roommate since 2010. +``` + +Paste this JSON into `Graph Schema` and keep the default extraction prompt. +It allows people and roommate relationships, with optional age and occupation. + +```json +{ + "vertexlabels": [ + {"id": 1, "name": "person", "id_strategy": "PRIMARY_KEY", + "primary_keys": ["name"], "properties": ["name", "age", "occupation"], + "nullable_keys": ["age", "occupation"]} + ], + "edgelabels": [ + {"name": "roommate", "source_label": "person", "target_label": "person", "properties": ["date"]} + ], + "propertykeys": [ + {"name": "name", "data_type": "TEXT", "cardinality": "SINGLE"}, + {"name": "age", "data_type": "INT", "cardinality": "SINGLE"}, + {"name": "occupation", "data_type": "TEXT", "cardinality": "SINGLE"}, + {"name": "date", "data_type": "TEXT", "cardinality": "SINGLE"} + ] +} +``` + +Follow these steps to build the indexes. For vector-only retrieval, stop after step 1. + +1. Click `Import into Vector` to split the document and build the chunk vector index. +2. Enter schema JSON or an existing graph name in `Graph Schema`, then click `Extract Graph Data (1)`. + Check the vertices, edges, and properties in `Output Info`. +3. Click `Load into GraphDB (2)` after checking the extraction. This writes to HugeGraph and refreshes vertex embeddings. +4. Open `Graph Tools`. Run `g.V().count()` and `g.E().count()`, then query the properties to confirm that the data was written. +{.steps} + +This example should produce 2 vertices and 1 edge, with Sarah's age set to `30` and occupation to `attorney`. +If the graph already contains data, use `Update Vid Embedding` to refresh its vertex index separately. +The `Graph Extraction Split Type` dropdown selects `document`, `paragraph`, or `sentence`. +`document` uses the whole input; the other options split it before extraction. The page can also inspect or clear chunk indexes, vertex indexes, and graph data. Clearing removes existing data, so first confirm that the current graph and indexes are not still used by other queries. @@ -37,15 +117,52 @@ Two collapsed helpers sit below the main controls: ## 2. GraphRAG Queries -The second tab can answer directly with the LLM, use only chunk-vector retrieval, use only graph retrieval, or combine graph and vector retrieval. +The second tab offers four answer modes. Choose the indexes needed for your question. + +| Control | Answer source | +| --- | --- | +| `Basic LLM Answer` | The LLM alone, without document or graph retrieval | +| `Vector-only Answer` | Retrieved document chunks | +| `Graph-only Answer` | Retrieved graph data | +| `Graph-Vector Answer` | Retrieved graph data and document chunks | + +Using `Graph-Vector Answer` is recommended, as it combines graph traversal and +vector search for more comprehensive fact retrieval. Other scopes are summarized in the table above. + +The screenshots use these short prompts. Paste them into `Query Prompt` and `Keywords Extraction Prompt`, respectively. +The service fills the placeholders. Keep the `KEYWORDS:` prefix and ASCII colons in keyword output, and preserve name capitalization. + +```text +Answer the question in English using only the retrieved context. Preserve names and state when the context is insufficient. +Context: {context_str} +Question: {query_str} +``` + +```text +Extract person names from the question and preserve their case. Return at most {max_keywords} names. +Return one line in this exact ASCII format: KEYWORDS:name:1.0 +Question: {question} +``` + +1. Enter `Who is Sarah?`, set `Graph-only Answer` to `False`, and set `Graph-Vector Answer` to `True`. +2. Keep `Template Num=-1` and `Rerank method=bleu` for the first check, then click `Answer Question`. +3. Read `Graph-Vector Answer` and compare facts such as age and occupation with the source document and graph. +{.steps} + +```gallery +![GraphRAG answer modes and a combined graph-vector answer](/images/docs/hugegraph-ai/en/graphrag.jpg) +![English prompts in GraphRAG](/images/docs/hugegraph-ai/en/query-prompts.jpg) +``` ```mermaid flowchart TD Q[Question] --> V[Query chunk vector index] Q --> K[Extract keywords] K --> M[Match graph vertices] - M --> T[Generate and execute Gremlin] - T -->|Failure| B[Fallback to BFS graph traversal] + M --> D{Text2Gremlin enabled} + D -->|Yes| T[Generate and execute Gremlin] + D -->|No| B[BFS graph traversal] + T -->|Failure| B T --> R[Prepare graph results] B --> R V --> S[Merge and rerank] @@ -58,7 +175,7 @@ Graph retrieval first matches HugeGraph vertices exactly by keyword and then use `Template Num` controls how Text2Gremlin participates in graph retrieval: - A negative value skips Text2Gremlin entirely, so graph retrieval goes straight to the predefined traversal. -- `0` generates Gremlin without any examples (zero-shot). +- `0` uses baseline generation without retrieving examples. - A positive value retrieves that many similar examples from the example index and uses the template-guided result. The example count is clamped to the range 0 to 10. Other controls on this tab are `Rerank method` (`bleu` or `reranker`), `Graph Ratio`, `Near neighbor first`, and `Query related information`, plus editable `Query Prompt` and `Keywords Extraction Prompt` fields. @@ -67,17 +184,57 @@ Below the single-question panel is a batch back-testing panel. Upload an `.xlsx` ## 3. Text2Gremlin -The third tab has two parts. The upper part builds the example vector index from a `.json` or `.csv` file of question and Gremlin pairs; the bundled `resources/demo/text2gremlin.csv` is used when no file is uploaded. +The third tab converts natural language into Gremlin and executes the generated queries. +The upper panel builds an example vector index from question and Gremlin pairs in `.json` or `.csv` files. +When no file is uploaded, it uses `resources/demo/text2gremlin.csv`. Match example labels and properties to the current graph. + +> [!TIP] +> If building the example index does not automatically retrieve similar templates, you can provide few-shot examples directly in the Prompt template below. + +Use these inputs to reproduce the Sarah query. + +1. Save the CSV below as `sarah-gremlin.csv`, upload it, and click `Build Example Vector Index`. Set `Number of refer examples` to `1`. +2. Paste the simplified schema below into `Schema`. It contains only the labels and properties needed for this query. +3. Paste the prompt below into `Prompt`, enter `Who is Sarah?`, then click `Text2Gremlin`. +4. Check both generated queries and their `Query ... Output` fields. This example returns Sarah with `age=30` and `occupation=attorney`. +{.steps} + +```csv +query,gremlin +Who is Sarah?,"g.V().has('person', 'name', 'Sarah').valueMap()" +``` -The lower part reads the graph schema, retrieves similar natural-language and Gremlin examples, fills the prompt with the question, schema, examples, and matched vertices, then generates Gremlin and optionally executes it. `Number of refer examples` sets how many examples are retrieved, from 0 to 10, and defaults to 2. The results appear in four fields: Gremlin with a template, Gremlin without a template, and the execution output for each. +```json +{"vertexlabels":[{"name":"person","properties":["name","age","occupation"]}],"edgelabels":[]} +``` -![RAG query scope selector](/images/docs/hugegraph-ai/quick-start-03.jpg) +```text +Translate the question into a read-only HugeGraph Gremlin query. +For person-name lookups, use g.V().has('person', 'name', '').valueMap(). +Preserve name capitalization. Query by name; do not use vertex IDs. +Example question: Who is Peter? +Example Gremlin: g.V().has('person', 'name', 'Peter').valueMap() +Graph schema: {schema} +Additional examples: {example} +Known vertices: {vertices} +Question: {query} +Return only one ```gremlin``` code block. +``` -A custom prompt must contain `{query}`, `{schema}`, `{example}`, and `{vertices}`. The REST API rejects a request if any placeholder is missing. +```gallery +![Generated Text2Gremlin queries and execution results](/images/docs/hugegraph-ai/en/text2gremlin.jpg) +``` + +A small model can generate incorrect IDs, change name capitalization, or use filters that do not meet the graph's index requirements. +Check the returned data before treating a query as successful. The Web UI executes generated queries directly, so use a dedicated test graph. +To generate a query without executing it, select the corresponding output types in the [REST API](./rest-api.md). + +A custom prompt must retain `{query}`, `{schema}`, `{example}`, and `{vertices}`. The REST API rejects it if any placeholder is missing. ## 4. Graph and Administration Tools `Graph Tools` runs a Gremlin query directly against the configured graph, triggers a manual graph backup, and can initialize demo data in HugeGraph through a beta action. A background job also backs up the graph every day at 01:00, and a second background task keeps vertex-id embeddings up to date while the process runs. +Initializing demo data first clears the current graph. Use this action only on a dedicated test graph. `Admin Tools` is password protected. Entering the configured `ADMIN_TOKEN` reveals the tail of `logs/llm-server.log`, which refreshes every 60 seconds, along with buttons to refresh or clear it. Access is refused while `ADMIN_TOKEN` is empty or still set to the placeholder `xxxx`. @@ -86,7 +243,9 @@ When `ENABLE_LOGIN=True`, the Web UI asks for basic credentials with the fixed u > [!WARNING] > In production, enable HugeGraph-LLM login, replace `USER_TOKEN` and `ADMIN_TOKEN`, and enforce a source IP allowlist at the firewall or network entry point. Separately enable [Server authentication and authorization](/docs/config/config-authentication/), retain Server audit logs (normally `audit-*.log`), and grant `GRAPH_USER` minimum required permissions. AI service tokens do not replace Server authentication. -![Keywords extracted in the RAG UI](/images/docs/hugegraph-ai/quick-start-04.png) +```gallery +![Handwritten Gremlin and its result in Graph Tools](/images/docs/hugegraph-ai/en/graph-tools.jpg) +``` ## 5. Prompt Language diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-pd.md b/content/en/docs/quickstart/hugegraph/hugegraph-pd.md index 4021420846..a62bce99a4 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-pd.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-pd.md @@ -123,7 +123,7 @@ PD startup reads installation `conf/application.yml`. Match this file to the ins #### 4.1 Configuration Reference -These tables were checked against `hg-pd-dist` configuration and Java defaults at Server master commit `2f827d6e8c9c62ae858f2fc122b3a192d015e2f4`, and the three `raft.rpc-*` rows against [apache/hugegraph#3368](https://github.com/apache/hugegraph/pull/3368); they do not describe every 1.7.0 behavior. Use original tagged configuration with release binaries and [master configuration](https://github.com/apache/hugegraph/blob/master/hugegraph-pd/hg-pd-dist/src/assembly/static/conf/application.yml) with master builds. +These tables list master configuration values and code defaults. See the [PDConfig implementation](https://github.com/apache/hugegraph/blob/master/hugegraph-pd/hg-pd-core/src/main/java/org/apache/hugegraph/pd/config/PDConfig.java) for Raft timeouts. Use the original tagged configuration with 1.7.0 binaries and [master configuration](https://github.com/apache/hugegraph/blob/master/hugegraph-pd/hg-pd-dist/src/assembly/static/conf/application.yml) with master builds. Differences from 1.7.0 are noted below. **gRPC and REST** diff --git a/content/en/docs/quickstart/hugegraph/hugegraph-server.md b/content/en/docs/quickstart/hugegraph/hugegraph-server.md index f1c899be91..91826ad341 100644 --- a/content/en/docs/quickstart/hugegraph/hugegraph-server.md +++ b/content/en/docs/quickstart/hugegraph/hugegraph-server.md @@ -84,7 +84,8 @@ on Java 24 or later, start Server with `-s false` (JDK 24+ dropped Security Mana serializer: className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1 config: - ioRegistries: [org.apache.hugegraph.structure.io.HugeGraphIoRegistry] + builder: org.apache.hugegraph.io.HugeGraphTypeSerializerRegistryBuilder + ioRegistries: [org.apache.hugegraph.io.HugeGraphIoRegistry] ``` ```yaml {tab="GraphSON (HTTP / Legacy)"} diff --git a/content/en/docs/quickstart/toolchain/hugegraph-hubble.md b/content/en/docs/quickstart/toolchain/hugegraph-hubble.md index 0893982740..0f3b81e6d9 100644 --- a/content/en/docs/quickstart/toolchain/hugegraph-hubble.md +++ b/content/en/docs/quickstart/toolchain/hugegraph-hubble.md @@ -12,49 +12,71 @@ and switch smoothly between graph, table, and JSON results. This guide uses **st For use with an HStore distributed cluster, read the foundational operations here first, then consult the [distributed supplement](/docs/quickstart/toolchain/visualization/hugegraph-hubble-hstore/). -This guide aligns with the Toolchain and Server `master` branches (both currently `1.8.0`). +This guide aligns with the Toolchain and Server `master` branches (currently based on 1.8, unreleased). Docker `latest` is a moving tag; use released versions for production deployments. + +The screenshots show the HugeGraph Hubble interface in action; features and workflows are identical in both English and Chinese. > [!WARNING] > Do not expose unauthenticated Hubble or Server instances directly to the public internet. In production, use HTTPS, container isolation, > [authentication and authorization](/docs/config/config-authentication/), and network allowlists. +Jump directly to a task below. If this is your first visit, start with the standalone deployment: + +- [Start and load a sample](#start-the-standalone-pair) + + Start the two services and load the People & Software Demo Graph. +- [Query and visualize](#gql-workspace-query-and-explore-relationships) + + Run Gremlin and inspect graph, table, or JSON results. +- [Import your own data](#data-source-management-prepare-the-input) + + Prepare CSV, map fields, and verify the imported records. +{.cards} + ## Start the standalone pair Use the main repository's [docker/docker-compose.yml](https://github.com/apache/hugegraph/blob/master/docker/docker-compose.yml) instead of writing custom Compose files. It already integrates RocksDB Server and Hubble with networking, health checks, and data volumes; see the adjacent [README](https://github.com/apache/hugegraph/blob/master/docker/README.md) for deployment details. -```bash -git clone --branch master --single-branch --depth 1 https://github.com/apache/hugegraph.git -cd hugegraph/docker -``` +1. **Prepare the official Compose configuration.** -If you already have the repository locally, navigate directly to its `docker/` directory. Compose automatically mounts -[`conf/hubble/standalone.properties`](https://github.com/apache/hugegraph/blob/master/docker/conf/hubble/standalone.properties), -which pre-configures `pd.enabled=false` and `server.direct_url=http://server:8080`. Because both services communicate over a shared Docker network, -you do not need to configure Server endpoints per graph. Start Compose directly from `docker/` so relative file paths resolve correctly. + ```bash + git clone --branch master --single-branch --depth 1 https://github.com/apache/hugegraph.git + cd hugegraph/docker + ``` -Hubble publishes on host loopback port `8088` by default (controlled by `HUBBLE_PUBLISH_HOST`, defaulting to `127.0.0.1`), whereas Server publishes -`8080` across all interfaces. For a local trial, set Server's `ports` in Compose to `127.0.0.1:8080:8080` to prevent exposing unauthenticated ports -beyond loopback (set `HUBBLE_PUBLISH_HOST=0.0.0.0` if remote access to Hubble is required). + If you already have the repository locally, navigate directly to its `docker/` directory. Compose automatically mounts + [`conf/hubble/standalone.properties`](https://github.com/apache/hugegraph/blob/master/docker/conf/hubble/standalone.properties), + which pre-configures `pd.enabled=false` and `server.direct_url=http://server:8080`. Because both services communicate over a shared Docker network, + you do not need to configure Server endpoints per graph. Start Compose directly from `docker/` so relative file paths resolve correctly. -Choose an unused project name for this trial and keep these variables in the same terminal session: + Hubble publishes on host loopback port `8088` by default (controlled by `HUBBLE_PUBLISH_HOST`, defaulting to `127.0.0.1`), whereas Server publishes + `8080` across all interfaces. For a local trial, set Server's `ports` in Compose to `127.0.0.1:8080:8080` to prevent exposing unauthenticated ports + beyond loopback (set `HUBBLE_PUBLISH_HOST=0.0.0.0` if remote access to Hubble is required). -```bash -export HUGEGRAPH_VERSION=latest -export HUBBLE_IMAGE=hugegraph/hubble:latest -export HUBBLE_DEMO_PROJECT="hubble-demo-$(date +%Y%m%d-%H%M%S)" -docker compose ls -docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml pull -docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml up -d --wait -docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml ps -curl -fsS http://127.0.0.1:8080/versions -``` +1. **Start and check the services.** + + Choose an unused project name for this trial and keep these variables in the same terminal session: -Once services are healthy, open . When `HUGEGRAPH_ADMIN_PASSWORD` is unset in `.env`, -Server permits anonymous access and Hubble opens the homepage directly. If authentication is enabled, configure an admin password and JWT secret in `.env` -as described in the Docker README, then sign in with a Server account (Hubble has no independent user database). -Profile and account management options appear only when authentication is enabled and the account holds requisite permissions. + ```bash + export HUGEGRAPH_VERSION=latest + export HUBBLE_IMAGE=hugegraph/hubble:latest + export HUBBLE_DEMO_PROJECT="hubble-demo-$(date +%Y%m%d-%H%M%S)" + docker compose ls + docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml pull + docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml up -d --wait + docker compose -p "$HUBBLE_DEMO_PROJECT" -f docker-compose.yml ps + curl -fsS http://127.0.0.1:8080/versions + ``` + +1. **Open Hubble.** + + Once services are healthy, open . When `HUGEGRAPH_ADMIN_PASSWORD` is unset in `.env`, + Server permits anonymous access and Hubble opens the homepage directly. If authentication is enabled, configure an admin password and JWT secret in `.env` + as described in the Docker README, then sign in with a Server account (Hubble has no independent user database). + Profile and account management options appear only when authentication is enabled and the account holds requisite permissions. +{.steps} Use `latest` to explore recent features; pin a released image version or digest for production. Official release archives are on the [download page](/docs/download/download/). @@ -153,22 +175,19 @@ the natural language prompt, or running statements in the code editor. ### Text2GQL natural-language queries {#text2gql} -> [!NOTE] -> The Text2GQL backend capabilities (including Model settings, built-in/temporary models, and Prompt templates) require a Hubble build with -> [apache/hugegraph-toolchain#824](https://github.com/apache/hugegraph-toolchain/pull/824) or a release containing it. -> Earlier builds display a preview tab without model management. - Describe your query requirement in the **Natural language** input. Text2GQL combines the current graph schema to generate read-only Gremlin or Cypher and execute it automatically. Generated statements can be edited freely (read-only safety checks still apply before execution). In **Async** mode, both generated and handwritten queries can be submitted as background tasks. Inputs automatically restore the graph's most recent query prompt without running it, and the execution history retains original natural-language questions. ```gallery -![Text2GQL query and graph results](/docs/images/hubble/text2gql.jpg) # Query workspace +![Text2GQL entry and handwritten query results](/docs/images/hubble/text2gql.jpg) # Unconfigured model; handwritten query ![Shared model settings](/docs/images/hubble/text2gql-settings.jpg) # Model settings ![Gremlin and Cypher Prompt templates](/docs/images/hubble/text2gql-prompt.jpg) # Text2GQL Prompt ``` +The screenshots show an unconfigured model. The graph results come from handwritten Gremlin; connect a working model before generating natural-language queries. + | Model source | Configuration and storage | | --- | --- | | Use built-in model | Configure under **System & Operations → Model settings**; Keys are encrypted in H2 and never returned to the browser | @@ -177,7 +196,7 @@ Inputs automatically restore the graph's most recent query prompt without runnin Choose **OpenAI compatible** for OpenAI, DeepSeek, GLM, and similar services, or **Ollama (local)** for local Ollama; unauthenticated Ollama needs no Key. The Base URL must be reachable from Hubble's backend; container `localhost` refers to that container. Public endpoints require HTTPS, while plain HTTP is accepted only for loopback or private-network models; when changing the Base URL in Model settings, supply a new Key or explicitly clear the saved Key. -Only the `admin` account (a super administrator in PD mode) manages shared settings. When Server authentication is disabled, anyone who can access the +Administrators with model-settings management permission manage shared settings. When Server authentication is disabled, anyone who can access the deployment can manage them. Under model settings, click **Text2GQL Prompt** in the header to customize Gremlin and Cypher generation rules and few-shot examples. @@ -209,7 +228,7 @@ For example, select K-neighbor (GET) with `source=1:marko`, `max_depth=1`, and ` After parameter validation, use the run button on the card or the form shortcut to explore one-hop neighbors. ![Built-in neighbor algorithm parameters and graph result](/docs/images/hubble/algorithms.jpg) -{width="2033" height="977"} +{width="1600" height="1000"} ## Async Tasks: confirm background results @@ -301,8 +320,8 @@ After changes, refresh permission context or sign in again and verify menus and Older Servers may hide or disable unsupported operations; a visible button alone does not establish resource authorization. See the [distributed supplement](/docs/quickstart/toolchain/visualization/hugegraph-hubble-hstore/) for space management. -![Authenticated account list in PD mode](/docs/images/hubble/accounts.jpg) -{width="2033" height="977"} +![Authenticated account list in standalone mode](/docs/images/hubble/accounts.jpg) +{width="1600" height="1000"} ## Operations: inspect the standalone Server @@ -353,19 +372,23 @@ docker compose -p "${HUBBLE_DEMO_PROJECT:?}" -f docker-compose.yml down --volume This removes the project's containers, network, named volumes, and anonymous volume, losing the sample data and Hubble import tasks. To retain data, omit `--volumes` when taking the deployment down and reuse the same project name when starting it again. -To obtain an exact master build, use JDK 17 and Maven. The Maven plugin installs the required Node/Yarn; -you do not need to install them separately. These commands skip tests: +Building current master requires **JDK 17, Maven 3.9+, Python 3.9+, and stable Node.js 24 (at least 24.22.0)**, +with **pnpm 12.14.0** selected through Corepack. Maven no longer installs Node/Yarn automatically. -```bash -git clone --branch master --single-branch https://github.com/apache/hugegraph-toolchain.git -cd hugegraph-toolchain -mvn install -pl hugegraph-client,hugegraph-loader -am -Dmaven.javadoc.skip=true -DskipTests -ntp -cd hugegraph-hubble -mvn package -Dmaven.javadoc.skip=true -DskipTests -ntp -cd apache-hugegraph-hubble-* -# Edit conf/hugegraph-hubble.properties with the correct Server URL -bin/start-hubble.sh -``` +1. Follow the [Java 17 migration instructions](https://github.com/apache/hugegraph-toolchain/blob/master/docs/java17-migration.md) + to install the Server SDK in an isolated Maven repository. +1. Follow the [Hubble source-build procedure](https://github.com/apache/hugegraph-toolchain/blob/master/hugegraph-hubble/README.md) + to install Python dependencies and build Client, Loader, and Hubble with that same repository (skipping tests). +1. Open the resulting `hugegraph-hubble/apache-hugegraph-hubble-*` directory, + configure Server connection settings in `conf/hugegraph-hubble.properties`, then start: + + ```bash + bin/start-hubble.sh -d + ``` +{.steps} + +Current master uses a new H2 metadata database. Retain the old Hubble distribution and its data, and use a separate data directory for the new version. +Old metadata has no automatic migration and cannot be reused directly. The bundled configuration binds to `localhost:8088`. `bin/stop-hubble.sh` requests graceful shutdown before forcing termination on timeout. For development and testing, see the [Toolchain local test guide](/docs/guides/toolchain-local-test/). diff --git a/content/en/docs/quickstart/toolchain/hugegraph-loader.md b/content/en/docs/quickstart/toolchain/hugegraph-loader.md index 5954f32778..4f08731bef 100644 --- a/content/en/docs/quickstart/toolchain/hugegraph-loader.md +++ b/content/en/docs/quickstart/toolchain/hugegraph-loader.md @@ -7,12 +7,110 @@ search_boost: 1.6 description: "Bulk import graph data into HugeGraph with Loader from files, HDFS, relational databases, Kafka, and other HugeGraph graphs." --- -This guide follows Toolchain `master` (currently `1.8.0`). Released packages and Docker `latest` may differ from source; -check the version you use and build from source for unreleased functionality. +This guide aligns with the Toolchain `master` branch (currently based on 1.8, unreleased). Released packages and Docker `latest` may differ; check your version before using unreleased features. + +### Quick start: local file import {#quick-local-file-import} + +Walk through a minimal import workflow: import two people and one relationship, then verify the data in Hubble. +First [install Loader](#2-get-hugegraph-loader) and start [HugeGraph Server](/docs/quickstart/hugegraph/hugegraph-server/). +The commands below run from the Loader directory, importing into the default graph `hugegraph`. +For an authenticated Server, append `--username` and `--password` (or `--token`). + +1. **Create the data files.** + + These CSV files have no header row. The mapping in the next step supplies the column names. + + ```bash + mkdir -p demo-loader + cat > demo-loader/people.csv <<'EOF' + docs-smoke-alice,29 + docs-smoke-bob,31 + EOF + cat > demo-loader/knows.csv <<'EOF' + docs-smoke-alice,docs-smoke-bob + EOF + ``` + +1. **Create the schema and mapping.** + + Dedicated `docs_*` names keep this example separate from the built-in `person` data. + File paths are relative to the directory where you run Loader. + + ```bash + cat > demo-loader/schema.groovy <<'EOF' + schema.propertyKey("docs_name").asText().ifNotExist().create() + schema.propertyKey("docs_age").asInt().ifNotExist().create() + schema.vertexLabel("docs_smoke_person").properties("docs_name", "docs_age") + .primaryKeys("docs_name").ifNotExist().create() + schema.edgeLabel("docs_smoke_knows").sourceLabel("docs_smoke_person") + .targetLabel("docs_smoke_person").ifNotExist().create() + EOF + cat > demo-loader/struct.json <<'EOF' + { + "version": "2.0", + "structs": [ + { + "id": "people", + "input": { + "type": "FILE", + "path": "demo-loader/people.csv", + "format": "CSV", + "header": ["docs_name", "docs_age"] + }, + "vertices": [{"label": "docs_smoke_person"}] + }, + { + "id": "knows", + "input": { + "type": "FILE", + "path": "demo-loader/knows.csv", + "format": "CSV", + "header": ["source_name", "target_name"] + }, + "edges": [{ + "label": "docs_smoke_knows", + "source": ["source_name"], + "target": ["target_name"], + "field_mapping": {"source_name": "docs_name", "target_name": "docs_name"} + }] + } + ] + } + EOF + ``` + +1. **Run the import.** + + Keep the same working directory. Change the host and port if Server runs elsewhere. + + ```bash + bin/hugegraph-loader.sh \ + --graphspace DEFAULT --graph hugegraph \ + --file demo-loader/struct.json --schema demo-loader/schema.groovy \ + --host 127.0.0.1 --port 8080 + ``` + + A first successful run imports two vertices and one edge. Check the success and failure counts in the summary. + +1. **Query the result in Hubble.** + + Connect Hubble to the same Server and graph, then run these queries separately in the Gremlin workspace. + + ```groovy + g.V().hasLabel('docs_smoke_person').valueMap('docs_name', 'docs_age') + g.V().has('docs_smoke_person', 'docs_name', 'docs-smoke-alice') + .out('docs_smoke_knows').values('docs_name') + ``` + + The first query returns Alice with age `29` and Bob with age `31`. The second returns `docs-smoke-bob`. +{.steps} + +For other inputs, continue with the [mapping reference](#33-write-data-source-mapping-file). +The [bundled full example](#4-complete-example) also includes software vertices and edge properties. ### 1 HugeGraph-Loader Overview -HugeGraph-Loader is the data import component of HugeGraph, which can convert data from various data sources into graph vertices and edges and import them into the graph database in batches. +HugeGraph-Loader converts source data into graph vertices and edges and imports them into HugeGraph in batches. Currently supported data sources include: - Local disk file or directory, supports TEXT, CSV and JSON format files, supports compressed files @@ -23,13 +121,9 @@ Currently supported data sources include: Local disk files and HDFS files support resumable uploads. -It will be explained in detail below. - > Note: HugeGraph-Loader requires HugeGraph Server service, please refer to [HugeGraph-Server Quick Start](/docs/quickstart/hugegraph/hugegraph-server) to download and start Server -> **Concurrency protection:** starting with HugeGraph 1.8.0, ordinary imports acquire same-Server ownership protection -> (requires a Server whose `GET .../mode/owners` returns `protocol: 1` and a matching Loader; current `master` does not include it yet). -> Older Servers retain the legacy import behavior described below. +> **Concurrent imports**: Loader on the master branch supports LOADING owner protection on the same Server instance when the Server's `GET .../mode/owners` returns `protocol: 1`. Servers without this endpoint retain legacy import behavior without this protection; see the compatibility notes below. > **Testing Guide**: For running HugeGraph-Loader tests locally, please refer to [HugeGraph Toolchain Local Testing Guide](/docs/guides/toolchain-local-test) @@ -81,24 +175,22 @@ A release may not include all `master` functionality; use the source build below #### 2.3 Clone source code to compile and install -Clone the `master` branch: +Building Loader from the master branch requires Java 17, Maven 3.6.3+, and Python 3.9+. Clone the repository and build: ```bash git clone --branch master --single-branch https://github.com/apache/hugegraph-toolchain.git -``` - -Compile and generate tar package: - -```bash cd hugegraph-toolchain -mvn clean package -pl hugegraph-loader -am -DskipTests -ntp +mvn package -pl hugegraph-client,hugegraph-loader -am -DskipTests -ntp +cd hugegraph-loader/apache-hugegraph-loader-* ``` +To build against candidate Server SDKs during development, refer to the [Java 17 migration guide](https://github.com/apache/hugegraph-toolchain/blob/master/docs/java17-migration.md). + For Oracle input, obtain a JDBC driver JAR compatible with your Oracle/JDK versions and place it in the extracted Loader `lib/` directory. The startup script loads JARs there; installing a driver only in the local Maven repository does not put it on Loader’s runtime classpath. ### 3 How to use -The basic process of using HugeGraph-Loader is divided into the following steps: +To import a dataset, prepare the schema, data and mapping, then run Loader. - Write graph schema - Prepare data files - Write input source map files @@ -106,10 +198,10 @@ The basic process of using HugeGraph-Loader is divided into the following steps: #### 3.1 Construct graph schema -This step is the modeling process. Users need to have a clear idea of ​​their existing data and the graph model they want to create, and then write the schema to build the graph model. +Define the vertex labels, edge labels and properties needed to represent your data before writing the schema. -For example, if you want to create a graph with two types of vertices and two types of edges, the vertices are "people" and "software", the edges are "people know people" and "people create software", and these vertices and edges have some attributes, For example, the vertex "person" has: "name", "age" and other attributes, -"Software" includes: "name", "sale price" and other attributes; side "knowledge" includes: "date" attribute and so on. +The example below uses `person` and `software` vertices, connected by `knows` and `created` edges. +People have names, ages and cities; software has a name and price. Each edge records a date.
Example graph with person and software vertices connected by knows and created edges @@ -117,7 +209,7 @@ For example, if you want to create a graph with two types of vertices and two ty
-After designing the graph model, we can use `groovy` to write the definition of `schema` and save it to a file, here named `schema.groovy`. +Save the schema definitions in `schema.groovy`. ```groovy // Create some properties @@ -138,7 +230,7 @@ schema.edgeLabel("knows").sourceLabel("person").targetLabel("person").properties schema.edgeLabel("created").sourceLabel("person").targetLabel("software").ifNotExist().create(); ``` -> Please refer to the corresponding section in [hugegraph-client](/docs/clients/hugegraph-client) for the detailed description of the schema. +> See [HugeGraph Client](/docs/clients/hugegraph-client) for the schema API. #### 3.2 Prepare data @@ -154,9 +246,9 @@ The data sources currently supported by HugeGraph-Loader include: ###### 3.2.1.1 Local disk file or directory -The user can specify a local disk file as the data source. If the data is scattered in multiple files, a certain directory is also supported as the data source, but multiple directories are not supported as the data source for the time being. +Set the input path to a local file or a directory. A single input source accepts one directory, not a list of directories. -For example, my data is scattered in multiple files, part-0, part-1 ... part-n. To perform the import, it must be ensured that they are placed in one directory. Then in the loader's mapping file, specify `path` as the directory. +For data split across `part-0`, `part-1` and other files, place those files in one directory and set the mapping's `path` to that directory. Supported file formats include: @@ -164,7 +256,8 @@ Supported file formats include: - CSV - JSON -TEXT is a text file with custom delimiters, the first line is usually the header, and the name of each column is recorded, and no header line is allowed (specified in the mapping file). Each remaining row represents a record, which will be converted into a vertex/edge; each column of the row corresponds to a field, which will be converted into the id, label or attribute of the vertex/edge; +TEXT uses a configurable delimiter. Each data row represents a record; its columns supply IDs, labels or properties. +Column names can come from a header row or from the mapping's `header` setting. An example is as follows: @@ -198,9 +291,10 @@ Currently supported compressed file types include: GZIP, BZ2, XZ, LZMA, SNAPPY_R The loader also supports some relational databases as data sources, and currently supports MySQL, PostgreSQL, Oracle, and SQL Server. -However, the requirements for the table structure are relatively strict at present. If **association query** needs to be done during the import process, such a table structure is not allowed. The associated query means: after reading a row of the table, it is found that the value of a certain column cannot be used directly (such as a foreign key), and you need to do another query to determine the true value of the column. +Loader does not follow foreign keys to query other tables while parsing a row. +Each input row must provide the values needed to construct its graph elements. Prepare any joins in the source query first. -For example, Suppose there are three tables, person, software and created +Consider three tables named `person`, `software` and `created`. ```text // person schema @@ -217,12 +311,14 @@ id | name | lang | price id | p_id | s_id | date ``` -If the id strategy of person or software is specified as PRIMARY_KEY when modeling (schema), choose name as the primary key (note: this is the concept of vertex-label in hugegraph), when importing edge data, the source vertex and target need to be spliced ​​out. For the id of the vertex, you must go to the person/software table with p_id/s_id to find the corresponding name. In the case of the schema that requires additional query, the loader does not support it temporarily. In this case, the following two methods can be used instead: +If `person` and `software` use `PRIMARY_KEY` IDs with `name` as the primary key, each edge needs its endpoints' names. +The `created` table contains only `p_id` and `s_id`; Loader will not query the names from those IDs. Use either of these ID designs. -1. The id strategy of person and software is still specified as PRIMARY_KEY, but the id column of the person table and software table is used as the primary key attribute of the vertex, so that the id can be generated by directly splicing p_id and s_id with the label of the vertex when importing an edge; -2. Specify the id policy of person and software as CUSTOMIZE, and then directly use the id column of the person table and the software table as the vertex id, so that p_id and s_id can be used directly when importing edges; +1. Keep `PRIMARY_KEY` and use each vertex table's `id` column as its primary key property. + Loader can then build endpoint IDs from `p_id`, `s_id` and the vertex labels. +2. Use custom IDs, taking each vertex table's `id` column as the vertex ID. The edge mapping can use `p_id` and `s_id` directly. -The key point is to make the edge use p_id and s_id directly, don't check it again. +Both designs let edge rows supply endpoint IDs without another lookup. ##### 3.2.2 Prepare vertex and edge data @@ -267,13 +363,11 @@ The edge data file consists of data line by line. Generally, each line is used a ##### 3.3.1 Mapping file overview -The mapping file of the input source is used to describe how to establish the mapping relationship between the input source data and the vertex type/edge type of the graph. It is organized in `JSON` format and consists of multiple mapping blocks, each of which is responsible for mapping an input source. Mapped to vertices and edges. +The mapping file uses JSON. Each item in `structs` contains one `input` and its `vertices` and `edges` mappings. -Specifically, each mapping block contains **an input source** and multiple **vertex mapping** and **edge mapping** blocks, and the input source block corresponds to the `local disk file or directory`, ` HDFS file or directory` and `relational database` are responsible for describing the basic information of the data source, such as where the data is, what format, what is the delimiter, etc. The vertex map/edge map is bound to the input source, which columns of the input source can be selected, which columns are used as ids, which columns are used as attributes, and what attributes are mapped to each column, the values ​​of the columns are mapped to what values ​​of attributes, and so on. +`input` describes the data location, format and delimiter. Vertex and edge mappings define labels, IDs, properties and field conversions. -In the simplest terms, each mapping block describes: where is the file to be imported, which type of vertices/edges each line of the file is to be used as which columns of the file need to be imported, and the corresponding vertices/edges of these columns. what properties, etc. - -> Note: The format of the mapping file before version 0.11.0 and the format after 0.11.0 has changed greatly. For the convenience of expression, the mapping file (format) before 0.11.0 is called version 1.0, and the version after 0.11.0 is version 2.0. And unless otherwise specified, the "map file" refers to version 2.0. +> Loader 0.11.0 introduced mapping format 2.0; earlier mappings use format 1.0. This page uses 2.0 unless stated otherwise. @@ -795,9 +889,7 @@ The nodes of vertex and edge mapping (a key in the JSON file) have a lot of the #### 3.4 Execute command import -After preparing the graph model, data file, and input source mapping relationship file, the data file can be imported into the graph database. - -The import process is controlled by commands submitted by the user, and the user can control the specific process of execution through different parameters. +With the schema, data and mapping ready, run Loader with the options below. ##### 3.4.1 Parameter description @@ -893,7 +985,7 @@ Copy a mapping mounted from a read-only Kubernetes ConfigMap to a writable volum ##### 3.4.2 Breakpoint Continuation Mode -Usually, the Loader task takes a long time to execute. If the import interrupt process exits for some reason, and next time you want to continue the import from the interrupted point, this is the scenario of using breakpoint continuation. +For a long import, use a progress file to resume after an interruption. Set `--incremental-mode` to `true` to resume from a progress file. Loader records progress when it closes normally after a successful import or a normal stop; an unexpected failure may prevent this write. @@ -905,12 +997,12 @@ order, which is the most recent one. > Note: A progress file is written after a successful import or a normal stop (including an error limit), regardless of `--incremental-mode` > or `--print-progress`. An import aborted by an unexpected failure may not write one. -If the data file formats are all legal and the import task is stopped by the user (CTRL + C or kill, kill -9 is not supported), that is to say, if there is no error record, the next import only needs to be set to -Continue for the breakpoint. +After stopping an import with `Ctrl+C` or `kill`, set `--incremental-mode true` to continue from the recorded position. +A forced stop with `kill -9` cannot save progress. -But if the limit of --max-read-errors, --max-parse-errors or --max-insert-errors is reached because too much data is invalid or network abnormality is reached, Loader will record these original rows that failed into -the failure file, after the user modifies the data lines in the failure file, set --failure-mode to true to import these "failure files" as input sources (does not affect the normal file import), -Of course, if there is still a problem with the modified data line, it will be logged again to the failure file (don't worry about duplicate lines, they are dropped when the file is closed). Failure mode lifts the three error limits above, so the whole failure file is scanned. +Invalid data or connection failures can exhaust `--max-read-errors`, `--max-parse-errors` or `--max-insert-errors`. +Correct the rows in the failure files, then use `--failure-mode true` to retry them. This leaves the original data files unchanged. +Rows that still fail are recorded again and deduplicated when the failure file closes. Failure mode lifts these three limits and scans the entire file. Each input source, that is each item of `structs` in the mapping file, gets its own failure file. The file is named after the `id` of that input source with the suffix `.error` and is stored in the `${struct}/failure-data` directory. Every failed line is written as a pair of lines: a tip line starting with `#### READ ERROR:`, `#### PARSE ERROR:` or `#### INSERT ERROR:`, followed by the original data line. When the input source has a header, that header is written as JSON to a sibling `${id}.header` file, so the failure file can be read back with the right columns. @@ -939,62 +1031,12 @@ The script runs the JVM under `JAVA_HOME` when that variable is set, and `java` #### 3.4.5 Minimal Local-File Import - -First start HugeGraph Server and ensure the current user can access `hugegraph` in graphspace `DEFAULT`, create schema, and write data. If authentication is enabled, add the appropriate `--username`, `--password`, or `--token`. This example requires only local files, without Kafka, HDFS, or JDBC. - -From the extracted Loader installation directory, create data, schema, and a version 2.0 mapping file: - -```bash -mkdir -p demo-loader - -cat > demo-loader/people.csv <<'EOF' -docs-smoke-alice,29 -docs-smoke-bob,31 -EOF - -cat > demo-loader/schema.groovy <<'EOF' -schema.propertyKey("docs_name").asText().ifNotExist().create() -schema.propertyKey("docs_age").asInt().ifNotExist().create() -schema.vertexLabel("docs_smoke_person").properties("docs_name", "docs_age").primaryKeys("docs_name").ifNotExist().create() -EOF - -cat > demo-loader/struct.json <<'EOF' -{ - "version": "2.0", - "structs": [ - { - "id": "people", - "input": { - "type": "FILE", - "path": "demo-loader/people.csv", - "format": "CSV", - "header": ["docs_name", "docs_age"] - }, - "vertices": [ - {"label": "docs_smoke_person"} - ], - "edges": [] - } - ] -} -EOF -``` - -The CSV has no header row; `input.header` in the mapping supplies column names. `input.path` is relative to the Loader process working directory, so keep running from the installation directory. If Server runs in another container, set `--host` to a Server address reachable by Loader: - -```bash -bin/hugegraph-loader.sh \ - --graphspace DEFAULT --graph hugegraph \ - --file demo-loader/struct.json --schema demo-loader/schema.groovy \ - --host 127.0.0.1 --port 8080 -``` - -Expected result: two vertices and zero edges imported. This example uses dedicated schema names to avoid conflicts with a preloaded `person` label. In Hubble, run `g.V().hasLabel('docs_smoke_person').values('docs_name')` to confirm `docs-smoke-alice` and `docs-smoke-bob`. +Follow the [local-file import steps](#quick-local-file-import) at the start of this page. #### 3.4.6 Concurrent imports and LOADING ownership {#loading-ownership} -Starting with HugeGraph 1.8.0, Loader supports concurrent import protection via the LOADING ownership protocol -(requires a Server whose `GET .../mode/owners` returns `protocol: 1` and a matching Loader; current `master` does not include it yet). +In the `master` branch, Loader implements a LOADING ownership protocol coordinated with Server's `/mode/owners` endpoint. +When the deployed Server provides this endpoint and returns `protocol: 1`, Loader enables ownership protection automatically; released Server 1.7.0 and older versions continue using the legacy mode switching flow. ```mermaid graph TD @@ -1031,7 +1073,8 @@ graph TD ### 4 Complete example -Given below is an example in the example directory of the hugegraph-loader package. ([GitHub address](https://github.com/apache/hugegraph-toolchain/tree/master/hugegraph-loader/assembly/static/example/file)) +The Loader distribution includes this dataset in `example/file/`. +See the [source files](https://github.com/apache/hugegraph-toolchain/tree/master/hugegraph-loader/assembly/static/example/file). #### 4.1 Prepare data @@ -1199,13 +1242,13 @@ count metrics ###### 4.5.1.1 Prepare data -If you just want to try out the loader, you can import the built-in example dataset without needing to prepare additional data yourself. +For a first import, use the built-in example dataset. -If using custom data, before importing data with the loader, we need to copy the data into the container. +For custom data, first copy the files into the Loader container. -First, following the steps in [4.1–4.3](#41-prepare-data), we can prepare the data and then use `docker cp` to copy the prepared data into the loader container. +Prepare the data, schema and mapping as described in [sections 4.1 to 4.3](#41-prepare-data), then copy them with `docker cp`. -Suppose we've prepared the corresponding dataset following the above steps, stored in the `hugegraph-dataset` folder with the following file structure: +The following example stores the files in `hugegraph-dataset`. ```bash tree -f hugegraph-dataset/ @@ -1230,9 +1273,9 @@ edge_created.json edge_knows.json schema.groovy struct.json vertex_person.cs ###### 4.5.1.2 Data loading -Taking the built-in example dataset as an example, we can use the following command to load the data. +Run this command to import the built-in dataset. -If you need to import your custom dataset, you need to modify the paths for `-f` (data script) and `-s` (schema) configurations. +For custom data, set `-f` to your mapping file and `-s` to your schema file. You can refer to [3.4.1-Parameter description](#341-parameter-description) for the rest of the parameters. @@ -1279,7 +1322,7 @@ meter metrics edge load rate(edges/s) : 53 ``` -You can also use `curl` or `hubble` to observe the import result. Here's an example using `curl`: +Query the imported data with Hubble or `curl`. These requests use the GraphSpace-aware Server API. For an older Server, omit `/graphspaces/DEFAULT` from the path. @@ -1292,9 +1335,9 @@ If you want to check the import result of edges, you can use `curl --compressed ##### 4.5.2 Enter the docker container to load data -Besides using `docker exec` directly for data import, we can also enter the container for data loading. The basic process is similar to [4.5.1](#451-use-docker-exec-to-load-data-directly). +You can also open a shell in the container and run the same import commands. -Enter the container by `docker exec -it loader bash` and execute the command: +Open the shell with `docker exec -it loader bash`, then run the import. ```bash sh bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/file/schema.groovy -h server -p 8080 @@ -1303,54 +1346,52 @@ sh bin/hugegraph-loader.sh -g hugegraph -f example/file/struct.json -s example/f The results of the execution will be similar to those shown in [4.5.1](#451-use-docker-exec-to-load-data-directly). #### 4.6 Import data by spark-loader -> The current source uses Spark 3.2.2 and Scala 2.12. Other combinations need independent verification. -> -The parameters of `spark-loader` are divided into two parts. Note: Because the abbreviations of -these two-parameter names have overlapping parts, please use the full name of the parameter. -And there is no need to guarantee the order between the two parameters. -- hugegraph parameters (Reference: [hugegraph-loader parameter description](https://hugegraph.apache.org/docs/quickstart/toolchain/hugegraph-loader/#341-parameter-description) ) -- Spark task submission parameters (Reference: [Submitting Applications](https://spark.apache.org/docs/3.3.0/submitting-applications.html#content)) -Example: -```bash -sh bin/hugegraph-spark-loader.sh --master yarn \ ---deploy-mode cluster --name spark-hugegraph-loader --file ./hugegraph.json \ ---username admin --token admin --host xx.xx.xx.xx --port 8093 \ ---graph graph-test --num-executors 6 --executor-cores 16 --executor-memory 15g -``` +The current `master` API sink uses Spark `3.5.8`, Scala `2.12` and Java 17 on the driver and executors. +The POM's Spark `3.2.2` dependency supplies compile-time APIs; it is not the runtime engine. +Set `SPARK_HOME` and `JAVA_HOME`, then use the distribution's shaded Loader JAR with its launcher. -`bin/hugegraph-spark-loader.sh` submits `org.apache.hugegraph.loader.spark.HugeGraphSparkLoader` through `${SPARK_HOME}/bin/spark-submit`, so `SPARK_HOME` has to point at a Spark installation. Every jar under `lib/` is put on the class path. The Spark application name defaults to `hugegraph-spark-loader` and can be changed through the `APP_NAME` environment variable. +Create the schema first and make data files readable by the executors. CSV and TEXT inputs need headerless UTF-8 data and an explicit mapping `header`. +Other charsets and `has_header: true` are rejected. Load vertex data before edge data when the edges depend on those vertices. +Replace the mapping path below with your prepared vertex mapping. -`bin/get-params.sh` splits the command line: only the options below are handed to the loader, every other argument is passed to `spark-submit` unchanged. The splitter matches long option names only, so short forms such as `-f` and `-g` are not recognised. - -```text ---graph --schema --host --port --username --token --protocol ---trust-store-file --trust-store-password --clear-all-data --clear-timeout ---incremental-mode --failure-mode --batch-insert-threads --single-insert-threads ---max-conn --max-conn-per-route --batch-size --max-parse-errors --max-insert-errors ---timeout --shutdown-timeout --retry-times --retry-interval --check-vertex ---print-progress --dry-run --sink-type --vertex-partitions --edge-partitions --help +```bash +bin/hugegraph-spark-loader.sh \ + --master 'local[2]' --driver-memory 1g \ + --file /path/to/vertex-mapping.json \ + --host 127.0.0.1 --port 8080 --graph hugegraph --sink-type true ``` -`--file` is treated separately: with `--deploy-mode cluster` the mapping file is shipped to the executors through `--files` and the loader receives only its base name, otherwise the path is passed through as written. - -In this mode the loader reads FILE, HDFS and JDBC input sources; a KAFKA or GRAPH input source is rejected. +Pass Spark options such as `--master`, `--conf` and `--jars` to the launcher alongside Loader options. +Additional JDBC drivers need explicit `--jars`; copying them into `lib/` does not distribute them to executors. +Query the target graph to verify counts and properties; the API sink's insertion counters do not report distributed writes. -By default (`--sink-type true`) each Spark partition opens a HugeClient and writes vertices and edges through the HugeGraph server API. With `--sink-type false` the loader generates HFiles and bulk-loads them into HBase instead, taking the table names and ZooKeeper settings from `--vertex-table-name`, `--edge-table-name`, `--hbase-zk-quorum`, `--hbase-zk-port`, `--hbase-zk-parent`, `--vertex-partitions` and `--edge-partitions`. +This path rejects `--incremental-mode true` and `--failure-mode true`. Standalone Spark supports client deployment only. +HBase bulkload (`--sink-type false`) is unsupported on Spark 3.5.8 / Java 17; keep an existing bulkload deployment on its separately validated +released Loader/JVM/Spark/HBase environment. +For details, see the [Spark migration guide](https://github.com/apache/hugegraph-toolchain/blob/master/hugegraph-loader/docs/spark-java17.md). #### 4.7 Import data by flink-cdc-loader -> The current source uses Flink 1.13.5 with flink-connector-mysql-cdc 2.2.1 and Scala 2.12. Other combinations need independent verification. - -`bin/hugegraph-flinkcdc-loader.sh` submits `org.apache.hugegraph.loader.flink.HugeGraphFlinkCDCLoader` through `${FLINK_HOME}/bin/flink run`, so `FLINK_HOME` has to be set. The job captures MySQL change events with Flink CDC and applies them to the graph, which keeps the graph in step with the source tables. +The current `master` integration targets Flink `2.2.1`, MySQL CDC `3.6.0-2.2` and Java 17. +Older examples using Flink 1.13.5 and CDC 2.2.1 belong to the previous implementation. +Set `FLINK_HOME`; the submission JVM, JobManager and TaskManager must all use Java 17. +Keep Flink's module-access options and exactly one shaded Loader JAR in the distribution's `lib/`. -The mapping file uses the same format as for the command-line loader, but every input source has to be a JDBC input source over MySQL: the loader takes `url`, `database`, `table`, `username` and `password` from it and parses the host and port out of `url`. Vertex and edge mappings work as usual. `--cdc-flush-interval` and `--cdc-sink-parallelism` in [3.4.1](#341-parameter-description) apply to this mode only. +Each mapping needs a MySQL JDBC input with `url`, `database`, `table`, `username` and `password`. +Set MySQL `binlog_row_image=FULL`. UPDATE events require explicit, nonempty `update_strategies` and unchanged graph IDs; +changes to vertex IDs, edge endpoints or sort keys are rejected. CDC also rejects `--dry-run true` and `--use-prefilter true`. -The command line is split by `bin/get-params.sh` in the same way as for `spark-loader`, with the arguments that are not loader options going to `flink run`. - -Example: +Without a separator, all arguments go to Loader. To set Flink options, put them before `--` and Loader options after it. +For a prepared mapping at `mysql-cdc.json`, submit with one sink writer. ```bash -sh bin/hugegraph-flinkcdc-loader.sh --file ./mysql-cdc.json \ ---host xx.xx.xx.xx --port 8080 --graph hugegraph --username admin --token admin +bin/hugegraph-flinkcdc-loader.sh -p 1 -- \ + --file ./mysql-cdc.json --host 127.0.0.1 --port 8080 \ + --graph hugegraph --cdc-sink-parallelism 1 ``` + +Each mapping uses a single source reader and sink writer. `--cdc-sink-parallelism` defaults to `1`; other values are rejected. +After `--`, Loader's `-p` means the Server port. Checkpoint flushing does not provide transactional graph writes or an exactly-once guarantee. +Validate edge CDC and broader mappings in your test environment before deploying to production. +For more details, see the [Flink CDC documentation](https://github.com/apache/hugegraph-toolchain/blob/master/hugegraph-loader/docs/flink-cdc-java17.md). diff --git a/content/en/docs/quickstart/toolchain/hugegraph-tools.md b/content/en/docs/quickstart/toolchain/hugegraph-tools.md index 18b2733589..c5f6fc9610 100644 --- a/content/en/docs/quickstart/toolchain/hugegraph-tools.md +++ b/content/en/docs/quickstart/toolchain/hugegraph-tools.md @@ -6,7 +6,7 @@ weight: 3 ### 1 HugeGraph-Tools Overview -HugeGraph-Tools is an automated deployment, management and backup/restore component of HugeGraph. +HugeGraph-Tools manages graphs and tasks and provides backup and restore commands. It also retains deployment commands for legacy Server/Studio versions. > **Testing Guide**: For running HugeGraph-Tools tests locally, please refer to [HugeGraph Toolchain Local Testing Guide](/docs/guides/toolchain-local-test) @@ -19,7 +19,7 @@ HugeGraph-Tools is included in the Toolchain distribution. You can download the #### 2.1 Download the compiled archive -Download the latest version of the HugeGraph-Toolchain package: +The following example uses the released HugeGraph-Toolchain `1.7.0` package. Check the [download page](/docs/download/download/) for releases. ```bash export VERSION=1.7.0 @@ -32,42 +32,83 @@ cd "${ARCHIVE}/apache-hugegraph-tools-incubating-${VERSION}" ``` #### 2.2 Clone source code to compile and install -Please ensure that the wget command is installed before compiling the source code -Download the latest version of the HugeGraph-Tools source package: +Choose one source below. Both build the released `1.7.0` Tools; install Maven and JDK 11 first. The archive route also requires `wget`. +Each example ends in the generated Tools directory, where `bin/hugegraph help` lists the available commands. -```bash -# 1. get from github -git clone https://github.com/apache/hugegraph-toolchain.git +```bash {tab="Git tag"} +git clone --branch 1.7.0 --single-branch https://github.com/apache/hugegraph-toolchain.git +cd hugegraph-toolchain +mvn package -pl hugegraph-tools -am -DskipTests -ntp +cd hugegraph-tools/apache-hugegraph-tools-incubating-1.7.0 +bin/hugegraph help +``` -# 2. Download a released source package +```bash {tab="Source archive"} export VERSION=1.7.0 export ARCHIVE="apache-hugegraph-toolchain-incubating-${VERSION}" wget "https://downloads.apache.org/hugegraph/${VERSION}/${ARCHIVE}-src.tar.gz" +tar zxf "${ARCHIVE}-src.tar.gz" +cd "${ARCHIVE}-src" +mvn package -pl hugegraph-tools -am -DskipTests -ntp +cd "hugegraph-tools/apache-hugegraph-tools-incubating-${VERSION}" +bin/hugegraph help ``` -Compile and generate tar package: +The build also writes `hugegraph-tools/target/apache-hugegraph-tools-incubating-1.7.0.tar.gz` relative to the source root. -```bash -cd hugegraph-toolchain -mvn package -pl hugegraph-tools -am -DskipTests -ntp -``` +#### 2.3 Build current source {#build-current-source} -The package is generated as `hugegraph-tools/target/apache-hugegraph-tools-${version}.tar.gz`, and the unpacked directory `hugegraph-tools/apache-hugegraph-tools-${version}` (containing `bin/` and `lib/`) is created next to it. +To use unreleased features from the master branch, build Tools from source (master is currently based on 1.8; requires JDK 17, Maven 3.6.3+, and Python 3.9+): +1. **Clone Toolchain source** + + ```bash + git clone --branch master --single-branch https://github.com/apache/hugegraph-toolchain.git + cd hugegraph-toolchain + ``` + +1. **Build matching Server SDK** + + Run the script below to download Server source and install the SDK into an isolated Maven repository: + + ```bash + java -version + mvn -version + candidate_dir=$(mktemp -d) + mkdir -p "$candidate_dir/m2" + bash hugegraph-client/assembly/travis/install-candidate-sdk.sh \ + master "$candidate_dir/server" "$candidate_dir/m2" + ``` + + See the [Java 17 migration guide](https://github.com/apache/hugegraph-toolchain/blob/master/docs/java17-migration.md) for details. + +1. **Package Tools** + + ```bash + mvn -Dmaven.repo.local="$candidate_dir/m2" \ + -Dsdk.validation.mode=candidate \ + package -pl hugegraph-tools -am -DskipTests -Dmaven.javadoc.skip=true -ntp + cd hugegraph-tools/apache-hugegraph-tools-* + bin/hugegraph help + ``` +{.steps} + +Keep the SDK repository while using this source build. The master build directory does not contain `incubating`. ### 3 How to use #### 3.1 Function overview -After decompression, enter the apache-hugegraph-tools-${version} directory, you can use `bin/hugegraph` or `bin/hugegraph help` to view the usage information, and `bin/hugegraph help ` to view the usage of a single sub-command. mainly divided: +From the extracted or generated Tools directory, run `bin/hugegraph help` for the command list, +or `bin/hugegraph help ` for one command. The commands cover these tasks. - Graph management type, graph-mode-set, graph-mode-get, graph-list, graph-get, graph-clear, graph-create, graph-clone and graph-drop - Asynchronous task management type, task-list, task-get, task-delete, task-cancel and task-clear - Gremlin type, gremlin-execute and gremlin-schedule - Backup/Restore type, backup, restore, migrate, schedule-backup and dump - Authentication data backup/restore type, auth-backup and auth-restore -- Install deployment type, deploy, clear, start-all and stop-all +- Legacy Server/Studio deployment, deploy, clear, start-all and stop-all ```bash Usage: hugegraph [options] [command] [command options] @@ -75,20 +116,21 @@ Usage: hugegraph [options] [command] [command options] ##### 3.2 [options]-Global Variable -`options` is a global variable of HugeGraph-Tools, which can be configured in hugegraph-tools/bin/hugegraph, including: +Set these global connection options on the command line or through the environment variables below. -- --graph，HugeGraph-Tools The name of the graph to operate on, the default value is hugegraph -- --url，The service address of HugeGraph-Server, the default is http://127.0.0.1:8080 -- --user，When HugeGraph-Server opens authentication, pass username -- --password，When HugeGraph-Server opens authentication, pass the user's password -- --timeout，Timeout when connecting to HugeGraph-Server, the default is 30s -- --trust-store-file，The path of the certificate file, when --url uses https, the truststore file used by HugeGraph-Client, the default is empty, which means using the built-in truststore file conf/hugegraph.truststore of hugegraph-tools -- --trust-store-password，The password of the certificate file, when --url uses https, the password of the truststore used by HugeGraph-Client, the default is empty, representing the password of the built-in truststore file of hugegraph-tools -- --throw-mode, whether HugeGraph-Tools throws the exception instead of printing the error message and exiting, the default is false (mainly used by tests) +- `--graph` selects the graph, default `hugegraph`. +- `--url` sets the Server address, default `http://127.0.0.1:8080`. +- `--user` supplies the username when Server authentication is enabled. +- `--password` supplies that user's password. +- `--timeout` sets the connection timeout, default 30 seconds. +- `--trust-store-file` selects the HTTPS truststore. An empty value uses the built-in `conf/hugegraph.truststore`. +- `--trust-store-password` supplies its password. An empty value uses the built-in truststore password. +- `--throw-mode` defaults to `false`, which prints errors and exits. Set it to `true` to throw exceptions, mainly for tests. -> The protocol is taken from the scheme of --url: use `https://...` to connect over https. --trust-store-file and --trust-store-password can only be set when --url uses https, and both --user and --password must be given together or omitted together. +> The scheme in `--url` selects the protocol. Use `https://...` for HTTPS and set truststore options only with that scheme. +> Supply both `--user` and `--password`, or omit both. -The above global variables can also be set through environment variables. One way is to use export on the command line to set temporary environment variables, which are valid until the command line is closed +Use `export` to set an option for the current shell and the commands launched from it. | Global Variable | Environment Variable | Example | @@ -115,7 +157,7 @@ export HUGEGRAPH_GRAPH=hugegraph ./bin/hugegraph gremlin-execute --script 'g.V().count()' ``` -Another way is to set the environment variable in the bin/hugegraph script: +You can also set these variables in `bin/hugegraph`. ```text #!/bin/bash @@ -152,7 +194,7 @@ Another way is to set the environment variable in the bin/hugegraph script: > graph-create, graph-clone, graph-clear and graph-drop raise --timeout to at least 300 seconds. -> When you need to restore the backup graph to a new graph, you need to set the graph mode to RESTORING mode; when you need to merge the backup graph into an existing graph, you need to first set the graph mode to MERGING model. +> Before restoring into a new graph, set its mode to `RESTORING`. To merge into an existing graph, use `MERGING`. ##### 3.4 Asynchronous task management Type，task-list、task-get、task-delete、task-cancel and task-clear @@ -210,7 +252,7 @@ Another way is to set the environment variable in the bin/hugegraph script: - --retry, specify the number of failed retries, the default is 3 - --thread-num or -T, the number of threads to use, default is Math.min(10, Math.max(4, CPUs / 2)) - -D, use the mode of -Dkey=value to specify dynamic parameters, which are used to specify HDFS configuration items when restoring graphs from HDFS, for example: -Dfs.default.name=hdfs://localhost:9000 - > restore command can be used only if --format is executed as backup for json + > `restore` accepts backups made with `--format json`. > restore requires the graph to be in RESTORING or MERGING mode (set it with graph-mode-set first), otherwise the command fails - migrate, migrate the currently connected graph to another HugeGraphServer - --target-graph, the name of the target graph, the default is hugegraph @@ -262,6 +304,12 @@ Another way is to set the environment variable in the bin/hugegraph script: ##### 3.8 Install the deployment type +Note: These deployment commands only apply to legacy versions (Server 0.6.1~0.10.4 and Studio 0.6.1~0.10.0), +and do not support current HugeGraph Server or Hubble. +For modern deployments, follow [Server Quick Start](/docs/quickstart/hugegraph/hugegraph-server/) +and [Hubble Quick Start](/docs/quickstart/toolchain/hugegraph-hubble/). +The options below are retained only for maintaining legacy environments. + - deploy, one-click download, install and start HugeGraph-Server and HugeGraph-Studio - -v, required, specifies the HugeGraph-Server and HugeGraph-Studio version to install, must be one of the versions listed in bin/version-map.yaml (0.6, 0.7, 0.8, 0.9, 0.10), which maps it to the matching server and studio release versions - -p, required, specifies the installed HugeGraph-Server and HugeGraph-Studio directories @@ -275,11 +323,12 @@ Another way is to set the environment variable in the bin/hugegraph script: > deploy, start-all, clear and stop-all are handed by `bin/hugegraph` straight to the shell scripts `bin/deploy.sh`, `bin/start-all.sh`, `bin/clear.sh` and `bin/stop-all.sh`, so the global options and environment variables in 3.2 do not apply to them. -> There is an optional parameter -u in the deploy command. When provided, the specified download address will be used instead of the default download address to download the tar package, and the address will be written into the `~/hugegraph-download-url-prefix` file; if no address is specified later When -u and `~/hugegraph-download-url-prefix` are not specified, the tar package will be downloaded from the address specified by `~/hugegraph-download-url-prefix`; if there is neither -u nor `~/hugegraph-download-url-prefix`, it will be downloaded from the default download address `https://github.com/hugegraph` +> `deploy -u` overrides the download URL and saves it in `~/hugegraph-download-url-prefix`. +> Later runs without `-u` reuse the saved URL. Without either setting, the script uses `https://github.com/hugegraph`. ##### 3.9 Specific command parameters -The specific parameters of each subcommand are as follows: +The help output below lists each subcommand and its options. ```bash Usage: hugegraph [options] [command] [command options] diff --git a/content/en/docs/quickstart/toolchain/visualization/hugegraph-hubble-hstore.md b/content/en/docs/quickstart/toolchain/visualization/hugegraph-hubble-hstore.md index 277a737c48..ac7dcb2fa7 100644 --- a/content/en/docs/quickstart/toolchain/visualization/hugegraph-hubble-hstore.md +++ b/content/en/docs/quickstart/toolchain/visualization/hugegraph-hubble-hstore.md @@ -18,11 +18,21 @@ The guidance below focuses on connectivity and operations; ensure prerequisite c Docker `latest` is a moving tag. Screenshots assist in UI navigation; verify deployed versions via image digests and build details. Metrics and permission APIs vary by version. +The screenshots show the HugeGraph Hubble interface in action; features and workflows are identical in both English and Chinese. + ## Connect to a distributed cluster Hubble still manages graph data through the **Server graph API**. In distributed mode, it discovers Servers through PD and collects cluster information from PD and Store. Hubble does not read or write graph data directly in Store. +```mermaid +flowchart LR + H[Hubble] -->|Graph API| S[Server] + S -->|Graph reads and writes| T[Store] + H -.->|Discovery and cluster information| P[PD] + H -.->|Node metrics| T +``` + 1. **Generate Hubble configuration.** Prepare `.env` in the main repository's `docker/` directory following the Docker README. ```bash @@ -101,8 +111,8 @@ plus the storage capacity limit. These deployment quotas neither report current Select a space before opening a graph. Check the current graph after switching spaces, especially when spaces contain identically named graphs. The list reflects account access. If a space is missing, check membership permissions before creating more graphs. -![GraphSpace creation and advanced resource limits](/docs/images/hubble/graphspace.jpg) -{width="2048" height="984"} +![GraphSpace creation form and entry to advanced resource limits](/docs/images/hubble/graphspace.jpg) +{width="1600" height="1000"} ### Reuse Schema templates @@ -138,7 +148,7 @@ legacy custom roles on older Servers are not interchangeable with these presets. See [Server authentication and authorization](/docs/config/config-authentication/). ![GraphSpace members and access presets](/docs/images/hubble/members.jpg) -{width="2048" height="984"} +{width="1600" height="1000"} ## Locate problems through the cluster overview diff --git a/scripts/docs-refresh-audit.md b/scripts/docs-refresh-audit.md new file mode 100644 index 0000000000..53321c16a9 --- /dev/null +++ b/scripts/docs-refresh-audit.md @@ -0,0 +1,108 @@ +# 文档更新盘点与截图基准 + +本轮盘点全站当前文档，更新 Hubble 双语指南及会导致照做失败的高影响指引，并完成六组 P2/P3 页面的后续改造。 +公开路由、旧锚点、历史版本导航和语言切换保持可用。下表记录本轮处理范围，不代表全站命令均已运行。 + +## 全站盘点 + +统计范围为 `content/{en,cn}/docs/**/*.md`：英文 112 篇、中文 123 篇。 +多出的 11 篇中文全部为 0.2–0.11.2 的历史 changelog，不属于当前操作指南漏译。 +下表为有具体证据的优先项；未列出的页面不代表已经逐条验证所有命令。 + +| 优先级 | 页面（相对于两语言的 `docs/`） | 问题与处理 | +| --- | --- | --- | +| P1，本轮 | `quickstart/toolchain/hugegraph-hubble.md` 与 `visualization/hugegraph-hubble-hstore.md` | 重拍双语截图，统一数据与步骤；增加任务入口、启动 Steps 和分布式调用关系；修正源码构建要求、模型管理权限及旧元数据说明。 | +| P1，本轮 | `clients/gremlin-console.md` | 旧 GraphSON 类与当前 Server 配置不一致；使用配套 GraphBinary 配置，连接、查询和关闭按顺序呈现，明确 Driver 调用与 Console 远程提交的区别。 | +| P1，本轮 | `quickstart/client/hugegraph-client.md` | 旧 JDK 说明与 master 的 Java 17 目标冲突；区分已发布依赖与源码版，完整示例采用 Tabs，补充保存与运行入口。 | +| P1，本轮 | `quickstart/hugegraph/hugegraph-server.md` | 关联 GraphBinary 示例使用了错误的 HugeGraph registry 包名；与官方配置对齐。 | +| P2，本轮 | `quickstart/toolchain/hugegraph-loader.md` | 靠前增加最短文件导入 Steps，保留映射参考和旧锚点；核实 Java 17、并发保护及 Spark/Flink 版本边界。真实导入 2 点 1 边并查询属性和邻居。 | +| P2，本轮 | `config/config-authentication.md` | 分清 run、Compose 与既有容器三条路径，初始化和重启使用 Steps；发布版与当前源码均验证三条路径的认证状态及原数据保留。 | +| P2，本轮 | `guides/backup-restore.md` | 先给模式选择表，再给备份、设置模式、恢复、还原原模式的闭环；真实对比 RESTORING/MERGING 的 Schema、属性、端点和 ID。 | +| P2，本轮 | `quickstart/hugegraph-ai/quick_start.md` | 重排 GraphRAG 与 Text2Gremlin，补充可复制数据、Schema 和 Prompt；本地模型真实抽取、入库、检索、生成并执行，重拍 8 张两语配对图，写清当前源码限制。 | +| P3，本轮 | `quickstart/toolchain/hugegraph-tools.md` | Git 标签与源码包用 Tabs 提供完整获取和构建路径，两路均实际构建并运行 help；旧 Studio deploy 单独标明版本范围。 | +| P3，本轮 | `guides/faq.md` | 按部署、查询、导入和建模组织可链接的问题标题，修正版本与故障判断；真实验证 gzip 解码和字符串 ID 的 URL 编码查询。 | + +## 固定来源 + +- Hubble：[官方 Toolchain master 快照](https://github.com/apache/hugegraph-toolchain/tree/d655b0372dfe189ebeff51ea885b3386a1bc4a2c/hugegraph-hubble)。 +- Server / PD / Store：[官方 Server master 快照](https://github.com/apache/hugegraph/tree/ebb8f4e17f0bf826cb50745c590d8bc54b45c6b6)。 +- Hubble 编译 SDK 使用仓库锁定的 [Java 17 candidate 基准](https://github.com/apache/hugegraph/tree/86c05ef9c5d417bd37eab21d4ecd5af82bab17ef)， + 与运行版本分别记录。两次 Server 提交间仅启动、容器脚本及其说明/测试有变化；运行的 Java 源码相同，启动脚本使用 master 版本。 +- HugeGraph-AI：[官方 main 快照](https://github.com/apache/hugegraph-ai/tree/da11249745180471420890fd5a44aee93bc33bae)。 +- Tools 获取路径使用已发布的 `1.7.0` 标签和官方源码包，当前源码能力另按 Toolchain 固定版本说明。 +- OINK：`go.mod` 固定的 `v1.3.1`，仅使用公开的 Cards、Steps、Tabs、Gallery、Mermaid 和代码折叠能力。 + +构建要求来自固定源码的实际脚本，不由旧版包或 `latest` 标签推断。 +Hubble 当前源码需要 JDK 17、Maven 3.9+、Python 3.9+、稳定 Node 24（至少 24.22.0）与 pnpm 12.14.0， +并使用同一独立 Maven 仓库构建配套 SDK、Client、Loader 和 Hubble。 + +## 双语截图约定 + +使用真实后端与生产前端构建，不拦截 API 返回模拟数据。采用相同视口、图、查询和页面状态， +通过 Hubble 自带语言切换分别截取 `zh-CN`、`en-US`，保留已有图片路径。 +图片来源、尺寸与 SHA-256 见 [截图清单](fixtures/hubble-screenshots.json)。 +普通截图为 1600 × 1000；Store 详情使用 1600 × 1200，以完整显示 Raft 与分区指标。 + +| 图片文件 | 真实状态与操作 | +| --- | --- | +| `overview.jpg` | `DEFAULT/hugegraph`，加载人物与软件示例后查看图卡。 | +| `schema.jpg` | 同一图的顶点类型，展示 `person` 主键与 `software` 数值 ID 策略。 | +| `query.jpg` | 执行 `g.V().hasLabel('person').outE('created').inV().path()`，展示实际图结果。 | +| `algorithms.jpg` | K-neighbor（GET），`source=1:marko`、`max_depth=1`、`limit=20`。 | +| `text2gql.jpg` | 模型未配置的真实工作台；图结果来自手写 Gremlin，不宣称自然语言生成成功。 | +| `text2gql-settings.jpg` | 模型尚未配置的设置页，不填入凭据或虚构可用模型。 | +| `text2gql-prompt.jpg` | 打开默认 Gremlin / Cypher Prompt 编辑界面，不调用外部模型。 | +| `accounts.jpg` | 已启用 Server 认证的单机账号管理。 | +| `graphspace.jpg` | PD 模式创建图空间的实际表单，显示高级资源入口。 | +| `members.jpg` | PD 模式下真实账号与图空间成员权限。 | +| `cluster-overview.jpg` | 实际 PD、Store、Server 的集群拓扑与采集状态。 | +| `store-node.jpg` | 同一 Store 的系统、磁盘、Raft 与分区指标。 | + +示例图为内置 `loader` 数据集，8 个顶点、6 条边。查询与算法截图应在额外导入 CSV 前完成， +使双语图规模一致。单机与分布式图数据目录独立，截图不混用两个模式的导航或权限入口。 + +更新时保留旧发行版图片，不能仅凭当前页面未引用就删除：历史版本可能仍需要它们。 +教程内的截图使用 Gallery；只有以图片为主体的独立相册才使用 Expo。 + +## 运行时发现的后续问题 + +Hubble 已有算法结果时切换界面语言，结果面板可能误显示“任务已提交”，且 Task ID 为空。 +本轮已实际复现；在目标语言重新执行相同算法后，能正常显示 4 个节点、3 条边的同步结果。 +截图采用重新执行后的真实状态。该问题属于 Toolchain 的结果状态判定，本轮仅更新文档，未修改 Hubble 产品代码。 + +## Hubble 等 P1 页面的验证 + +- 使用 CI 同版本的 Python 3.13、Hugo Extended 0.168.0、Dart Sass 1.105.2，以及本机 WebP 工具执行源码测试。 + 下载页生成结果测试在新构建产物上另行执行，避免其默认跳过。 +- 新鲜 latest 产物的严格构建与逐版本安全检查通过。构建使用 staging URL，历史版本指向已发布站点；未发布到远端。 +- `bash dist/validate-links.sh` 与 `tests/e2e/search-ranking.spec.js` 通过。 +- 两语 320 / 390 / 1440 像素布局、Cards 锚点、Steps、Tabs 键盘操作和旧示例深链接通过浏览器检查。 +- 全部 24 张图片均已实际加载，并逐张检查原图查看器、Escape 关闭和焦点恢复；Mermaid 已实际渲染。 +- Java 完整示例复制包含折叠内容，打印包含两个 Tab，Markdown 导出保留完整示例及图片链接。 +- Console 的 GraphBinary 与 traversal alias 已对真实 HStore Server 执行，两条计数查询均返回示例图的 8 个顶点。 + 中英文四份 Java 示例已使用真实发布版 `1.7.0` 依赖编译；未执行会修改图数据的完整 Java 示例。 +- 独立审查覆盖文字变更与 24 张截图；算法图重拍后复查通过。保留了桌面与移动端的改前、改后页面截图。 + +本轮没有修改共享主题样式或历史发布分支，未重新组装历史版本，也未执行完整历史聚合发布门禁。 + +## 六组后续页面的验证 + +- 从固定源码构建配套 Server、Client、Loader 和 Tools，使用独立图与数据验证操作。Loader 导入 2 个顶点和 1 条边，姓名、年龄和邻居查询均匹配。 +- 认证覆盖固定 1.7.0 镜像和当前源码构建，各自验证 run、Compose、既有容器离线启用三条路径。匿名或错误密码返回 401，正确密码返回 200；既有图的顶点 ID、类型和属性保留。 +- 备份源包含 PRIMARY_KEY 人物、AUTOMATIC 顶点及带属性的边。RESTORING 保留 Schema 和元素 ID，MERGING 按目标 Schema 重算主键 ID、保留 AUTOMATIC ID，边端点随之更新；两路均还原原图模式。 +- Tools 的 Git 标签与官方源码包分别真实构建，生成目录和 `bin/hugegraph help` 均正确。Loader 页的 Spark/Flink 说明核对了固定源码，未把它们写成已完成分布式作业验收。 +- FAQ 的 `curl --compressed` 真实接收 gzip 并解码 JSON；对返回的字符串 ID 进行完整 JSON 编码后，可以查询到原顶点。 +- HugeGraph-AI 使用本地 Ollama `qwen2.5:3b`、`mxbai-embed-large` 和 Faiss。两语使用同一份 Sarah / James 数据，真实抽取并写入 2 点 1 边。Text2Gremlin 的生成语句经 Server 执行，返回 Sarah 的年龄和职业，没有手改生成结果。 +- 中文按 writing-human 的教程规则复查动作、条件和可见结果，英文同步精简重复解释。旧锚点通过显式 ID 保留，技术字段和命令保持准确。 +- 最终新鲜产物通过 CI 对应工具链的源码检查、严格构建、安全检查、内部链接与搜索排名；浏览器覆盖六组双语、P1 回归、移动端导航、完整复制及 Print/Markdown 回退。 +- 独立只读审查提出的版本、仓库参数、模式还原和表格模板问题均已修复并复审。AI 六列表格经两语页面实际下载、上传和预览，未把输入模板当作批量生成成功。 + +AI 截图的来源、模型、输入、状态和 SHA-256 见[截图清单](fixtures/ai-screenshots.json)。 +AI 的 `LANGUAGE` 切换提示词和示例，控件仍为英文；配图保留实际界面，未翻译图片像素。旧图片保留供历史内容使用。 + +## AI 源码限制与后续项 + +- 固定源码缺少 `questions_template.xlsx` 时，Web 页面无法启动。教程提供创建真实输入模板的步骤，未改产品代码。 +- Text2Gremlin 的自动示例召回在本轮未通过，`Similar Template (TopN)` 为空。教程使用可复制 Prompt 直接提供参考示例，配图展示真实生成和执行，不宣称自动召回成功。 +- 小模型会生成错误 ID、改变大小写或产生不满足索引条件的查询。教程保留检查结果的步骤，不承诺每次生成正确。 +- Graph-only 的中文流曾出现空输出。复现主流程采用两语均验证的 Graph-Vector 范围；四种范围表说明控件用途，不代表四条路径均已通过。 diff --git a/scripts/fixtures/ai-screenshots.json b/scripts/fixtures/ai-screenshots.json new file mode 100644 index 0000000000..0544712f41 --- /dev/null +++ b/scripts/fixtures/ai-screenshots.json @@ -0,0 +1,114 @@ +{ + "source": { + "repo": "apache/hugegraph-ai", + "ref": "main", + "sha": "da11249745180471420890fd5a44aee93bc33bae" + }, + "server": { + "runtime_sha": "ebb8f4e17f0bf826cb50745c590d8bc54b45c6b6", + "sdk_sha": "86c05ef9c5d417bd37eab21d4ecd5af82bab17ef", + "graph": "aidocs", + "graphspace": "DEFAULT", + "backend": "memory", + "vertices": 2, + "edges": 1 + }, + "models": [ + { + "name": "qwen2.5:3b", + "digest": "357c53fb659c5076de1d65ccb0b397446227b71a42be9d1603d46168015c9e4b" + }, + { + "name": "mxbai-embed-large:latest", + "digest": "468836162de7f81e041c43663fedbbba921dcea9b9fefea135685a39b2d83dd8", + "embedding_dimension": 1024 + } + ], + "model_server": "Ollama v0.40.8; 127.0.0.1:18200; local inference only", + "language": "EN/CN changes prompts; all Gradio control labels remain English", + "data": "Sarah is a 30-year-old attorney. James is a journalist. Sarah has been James's roommate since 2010.", + "question": "Who is Sarah?", + "rag_state": { + "raw": false, + "vector_only": false, + "graph_only": false, + "graph_vector": true, + "template_num": -1, + "graph_ratio": 0.6, + "rerank": "bleu" + }, + "text2_state": { + "example_num": 1, + "similar_template": [], + "prompt_example": "Peter supplied directly in Prompt", + "actual_generation": "g.V().has('person', 'name', 'Sarah').valueMap()", + "output": "Sarah; age 30; occupation attorney; both generated queries executed" + }, + "graph_tools_query": "g.V().valueMap()", + "screenshots": [ + { + "file": "static/images/docs/hugegraph-ai/cn/graph-tools.jpg", + "locale": "cn", + "width": 1472, + "height": 607, + "sha256": "3f7c1b35f6fda8b98e5099aac1f28ec90d4a9d810c64000c22e4ddd62ed5cb11", + "capture": "Playwright Chromium native screenshot; no API interception or pixel edits" + }, + { + "file": "static/images/docs/hugegraph-ai/cn/graphrag.jpg", + "locale": "cn", + "width": 1482, + "height": 671, + "sha256": "9066845ce4897c5642f4a2c3987c4781795d93033b09d64f71340aed7767fffb", + "capture": "Playwright Chromium native screenshot; no API interception or pixel edits" + }, + { + "file": "static/images/docs/hugegraph-ai/cn/query-prompts.jpg", + "locale": "cn", + "width": 968, + "height": 425, + "sha256": "e0a90b23d1df828f8b552511476f6c70a4465a739d8d781974fecfe54ecb19f1", + "capture": "Playwright Chromium native screenshot; no API interception or pixel edits" + }, + { + "file": "static/images/docs/hugegraph-ai/cn/text2gremlin.jpg", + "locale": "cn", + "width": 1472, + "height": 1317, + "sha256": "ac3add4c661080cd461925f43c17e328f82aa1c78387b61101cf13fabb0d2e98", + "capture": "Playwright Chromium native screenshot; no API interception or pixel edits" + }, + { + "file": "static/images/docs/hugegraph-ai/en/graph-tools.jpg", + "locale": "en", + "width": 1472, + "height": 607, + "sha256": "3f7c1b35f6fda8b98e5099aac1f28ec90d4a9d810c64000c22e4ddd62ed5cb11", + "capture": "Playwright Chromium native screenshot; no API interception or pixel edits" + }, + { + "file": "static/images/docs/hugegraph-ai/en/graphrag.jpg", + "locale": "en", + "width": 1482, + "height": 671, + "sha256": "75ff18d9eb2488dc62d5f586146b772d0416a841a386c6f8ef3626f50262d9ea", + "capture": "Playwright Chromium native screenshot; no API interception or pixel edits" + }, + { + "file": "static/images/docs/hugegraph-ai/en/query-prompts.jpg", + "locale": "en", + "width": 968, + "height": 425, + "sha256": "aab594a5f0e1ccb3e56a0f351debdcd80f1590c7bc7a8704865909fc494561f6", + "capture": "Playwright Chromium native screenshot; no API interception or pixel edits" + }, + { + "file": "static/images/docs/hugegraph-ai/en/text2gremlin.jpg", + "locale": "en", + "width": 1472, + "height": 1317, + "sha256": "14fe41a4c2da7da8fa1cc102fac9571625bcc9eaa486a06dcd16eca5998feddf", + "capture": "Playwright Chromium native screenshot; no API interception or pixel edits" + } + ] +} diff --git a/scripts/fixtures/hubble-screenshots.json b/scripts/fixtures/hubble-screenshots.json new file mode 100644 index 0000000000..92bcd4623e --- /dev/null +++ b/scripts/fixtures/hubble-screenshots.json @@ -0,0 +1,184 @@ +{ + "capturedOn": "2026-10-22", + "hubbleSource": "apache/hugegraph-toolchain@d655b0372dfe189ebeff51ea885b3386a1bc4a2c", + "serverRuntimeSource": "apache/hugegraph@ebb8f4e17f0bf826cb50745c590d8bc54b45c6b6", + "compileSdkSource": "apache/hugegraph@86c05ef9c5d417bd37eab21d4ecd5af82bab17ef", + "capture": "Real source-built services; native language toggle; no mocked API responses or external model calls", + "sample": { + "dataset": "loader", + "graphspace": "DEFAULT", + "graph": "hugegraph", + "vertices": 8, + "edges": 6 + }, + "images": [ + { + "path": "content/en/docs/images/hubble/accounts.jpg", + "language": "en", + "width": 1600, + "height": 1000, + "sha256": "41d523cac35c6d2ce63d3485d0811aa83d736df09358902f516b39666e98b1b5" + }, + { + "path": "content/en/docs/images/hubble/algorithms.jpg", + "language": "en", + "width": 1600, + "height": 1000, + "sha256": "604eeb07917bc7ac23162bb09d8f5086253014ea81126460ee21a4af2dbdd059" + }, + { + "path": "content/en/docs/images/hubble/cluster-overview.jpg", + "language": "en", + "width": 1600, + "height": 1000, + "sha256": "cd05d3c6dce882f788c83ce136a5def56215cc40857041b4be1dfec8c22d77ac" + }, + { + "path": "content/en/docs/images/hubble/graphspace.jpg", + "language": "en", + "width": 1600, + "height": 1000, + "sha256": "1d18a7b25fb360598836d89e32a1aed4219386df765b6b0739cca1d42b0958a0" + }, + { + "path": "content/en/docs/images/hubble/members.jpg", + "language": "en", + "width": 1600, + "height": 1000, + "sha256": "a88066b474d0367efebe7ab6a49fc82c4ca8594e98080dfa23910ef9c3240c77" + }, + { + "path": "content/en/docs/images/hubble/overview.jpg", + "language": "en", + "width": 1600, + "height": 1000, + "sha256": "00fb9ce86de1f19c15efdd5008a350a7ded4264bffc356c46e3c589dab293020" + }, + { + "path": "content/en/docs/images/hubble/query.jpg", + "language": "en", + "width": 1600, + "height": 1000, + "sha256": "79d95e34de55e722feb18e528eba3539bbf54abfc27701d3b72619b9e0a8194c" + }, + { + "path": "content/en/docs/images/hubble/schema.jpg", + "language": "en", + "width": 1600, + "height": 1000, + "sha256": "2f84c77f5d413d88d63443d4cdbec684e68f4aebf17aac57a528c03adbb362fe" + }, + { + "path": "content/en/docs/images/hubble/store-node.jpg", + "language": "en", + "width": 1600, + "height": 1200, + "sha256": "18898b4f487a11835eb9a46fdfb0b86617cbcd6304d3cf9d95c7dd032d3f20be" + }, + { + "path": "content/en/docs/images/hubble/text2gql-prompt.jpg", + "language": "en", + "width": 1600, + "height": 1000, + "sha256": "18983352307f1a4dbeb0ed4d23c97f268d00a6e7d14ce3dfcf77a7f9a375872d" + }, + { + "path": "content/en/docs/images/hubble/text2gql-settings.jpg", + "language": "en", + "width": 1600, + "height": 1000, + "sha256": "f09e346087cf7f3697161e459cbaf25516badb4f7198eaff66b28cc123dce581" + }, + { + "path": "content/en/docs/images/hubble/text2gql.jpg", + "language": "en", + "width": 1600, + "height": 1000, + "sha256": "889730977372a841491e99ef054b6c470f19b9337d2b42ec3690e8a05e04e545" + }, + { + "path": "content/cn/docs/images/hubble/accounts.jpg", + "language": "cn", + "width": 1600, + "height": 1000, + "sha256": "37b43141942d27e0f004e6a42d21225884fbd2cc5553510a07e80bfdb75796c8" + }, + { + "path": "content/cn/docs/images/hubble/algorithms.jpg", + "language": "cn", + "width": 1600, + "height": 1000, + "sha256": "1b7c06a02e1f029c688d031dd255717b2fb8019a8fb2d7de5a6adb073562ec81" + }, + { + "path": "content/cn/docs/images/hubble/cluster-overview.jpg", + "language": "cn", + "width": 1600, + "height": 1000, + "sha256": "1585fa02d2d4f4d0f5eaca703116d9109ddac4172bc00f70b148bcece4a5bb8a" + }, + { + "path": "content/cn/docs/images/hubble/graphspace.jpg", + "language": "cn", + "width": 1600, + "height": 1000, + "sha256": "50cac0d30fb884e6fedd49dfb7dfa6aa2f3e9f604d38aaf6dfb6db1cf1500dec" + }, + { + "path": "content/cn/docs/images/hubble/members.jpg", + "language": "cn", + "width": 1600, + "height": 1000, + "sha256": "261d02a39f52780a7f3cbfbd07b88d26b9c73f5b18a77e03f58d441f7bc4bdce" + }, + { + "path": "content/cn/docs/images/hubble/overview.jpg", + "language": "cn", + "width": 1600, + "height": 1000, + "sha256": "bb2adeef49baa37530e75509d089f3ac93f095def6e52b09f5ff9b25ae05b0d4" + }, + { + "path": "content/cn/docs/images/hubble/query.jpg", + "language": "cn", + "width": 1600, + "height": 1000, + "sha256": "419cf0a28d1748db7edbe314425a1958ded5f11843da500e7624756cb07ef2e5" + }, + { + "path": "content/cn/docs/images/hubble/schema.jpg", + "language": "cn", + "width": 1600, + "height": 1000, + "sha256": "dd5be5c084750ace80612b88642d50a356d85e16951995d3c193a0a4030cf4fc" + }, + { + "path": "content/cn/docs/images/hubble/store-node.jpg", + "language": "cn", + "width": 1600, + "height": 1200, + "sha256": "44ae79b28d0b933f7861e0500ac71d37adf6a3149b371bf9401ee8881c157908" + }, + { + "path": "content/cn/docs/images/hubble/text2gql-prompt.jpg", + "language": "cn", + "width": 1600, + "height": 1000, + "sha256": "a9c8535bc7cb967192d5b146ba822334908d696314d4bb7bb421e3dc5d984495" + }, + { + "path": "content/cn/docs/images/hubble/text2gql-settings.jpg", + "language": "cn", + "width": 1600, + "height": 1000, + "sha256": "38f81de3c7ce1880638f3a490aae709200b87d1de857269d95b5a60451105b9b" + }, + { + "path": "content/cn/docs/images/hubble/text2gql.jpg", + "language": "cn", + "width": 1600, + "height": 1000, + "sha256": "af498fb297cf8477174cb62d7aaae0bcc7127eee7f39c09e082bb9d9dc2d17b4" + } + ] +} diff --git a/static/images/docs/hugegraph-ai/cn/graph-tools.jpg b/static/images/docs/hugegraph-ai/cn/graph-tools.jpg new file mode 100644 index 0000000000..353a1e5b27 Binary files /dev/null and b/static/images/docs/hugegraph-ai/cn/graph-tools.jpg differ diff --git a/static/images/docs/hugegraph-ai/cn/graphrag.jpg b/static/images/docs/hugegraph-ai/cn/graphrag.jpg new file mode 100644 index 0000000000..3508487cfa Binary files /dev/null and b/static/images/docs/hugegraph-ai/cn/graphrag.jpg differ diff --git a/static/images/docs/hugegraph-ai/cn/query-prompts.jpg b/static/images/docs/hugegraph-ai/cn/query-prompts.jpg new file mode 100644 index 0000000000..45bb8240e6 Binary files /dev/null and b/static/images/docs/hugegraph-ai/cn/query-prompts.jpg differ diff --git a/static/images/docs/hugegraph-ai/cn/text2gremlin.jpg b/static/images/docs/hugegraph-ai/cn/text2gremlin.jpg new file mode 100644 index 0000000000..375eeaab37 Binary files /dev/null and b/static/images/docs/hugegraph-ai/cn/text2gremlin.jpg differ diff --git a/static/images/docs/hugegraph-ai/en/graph-tools.jpg b/static/images/docs/hugegraph-ai/en/graph-tools.jpg new file mode 100644 index 0000000000..353a1e5b27 Binary files /dev/null and b/static/images/docs/hugegraph-ai/en/graph-tools.jpg differ diff --git a/static/images/docs/hugegraph-ai/en/graphrag.jpg b/static/images/docs/hugegraph-ai/en/graphrag.jpg new file mode 100644 index 0000000000..46590ce686 Binary files /dev/null and b/static/images/docs/hugegraph-ai/en/graphrag.jpg differ diff --git a/static/images/docs/hugegraph-ai/en/query-prompts.jpg b/static/images/docs/hugegraph-ai/en/query-prompts.jpg new file mode 100644 index 0000000000..168af92768 Binary files /dev/null and b/static/images/docs/hugegraph-ai/en/query-prompts.jpg differ diff --git a/static/images/docs/hugegraph-ai/en/text2gremlin.jpg b/static/images/docs/hugegraph-ai/en/text2gremlin.jpg new file mode 100644 index 0000000000..56dafa891a Binary files /dev/null and b/static/images/docs/hugegraph-ai/en/text2gremlin.jpg differ