Elasticsearch:使用jdbc-rivers设置父/子

 micheals 发布于 2023-02-05 08:26

我正在使用jdbc-river从Sql Server数据库/表中读取数据.截至目前,我已为数据库中的每个表创建了一个单独的类型.作为我实现的下一步,我想使用父/子类型,以便我可以翻译我的sql表之间的关系并存储它们.

Table1
Col_id| name| prop1|prop2|prop3

child_table1
col_id| table_id| child_prop1|child_prop2|child_prop3


curl -XPUT 'localhost:9200/_river/parent/_meta' -d '{
    "type" : "jdbc",
    "jdbc" : {
        "driver" : "com.mysql.jdbc.Driver",
        "url" : "jdbc:mysql://localhost:3306/test",
        "user" : "",
        "password" : "",
        "sql" : "select * from table1",
        "index" : "index1",
        "type" : "parent"
    }
}'

curl -XPUT 'localhost:9200/_river/child/_meta' -d '{
    "type" : "jdbc",
    "jdbc" : {
        "driver" : "com.mysql.jdbc.Driver",
        "url" : "jdbc:mysql://localhost:3306/test",
        "user" : "",
        "password" : "",
        "sql" : "select * from child_table1",
        "index" : "index1",
        "type" : "child"
    }
}'



curl -XPOST 'localhost:9200/_river/child/_mapping' -d '{
  "child":{
    "_parent": {"type": "parent"}
  }
}'

我想以下列格式存储我的数据

{
  "id": "1",
  "name": "A leading wordsmith",
  "prop1": "data",
  "prop2": "data",
  "prop3": "data",

  "child": [
    {
      "child_prop1": "data",
      "child_prop2": "data",
      "child_prop3": "data",
    }
    {
      "child_prop1": "data1",
      "child_prop2": "data1",
      "child_prop3": "data1",
    }
  ]
}

任何人都可以评论我如何使用jdbc-rivers将我的数据存储为上述场景的父/子类型.

更新 根据以下反馈,更新的映射和元.

curl -XPOST 'http://localhost:9200/library' -d '{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0
  },
  "mappings": {
    "person": {
      "properties": {
        "person_id": {
          "type": "integer"
        },
        "name": {
          "type": "string"
        }
      }
    },
    "work": {
      "_parent": {
        "type": "person"
      },
      "properties": {
        "person_id": {
          "type": "integer",
          "index": "not_analyzed"
        },
        "name": {
          "type": "string"
        },
        "genre": {
          "type": "string"
        },
        "publisher": {
          "type": "string"
        }
      }
    }
  }
}'

curl -XPUT localhost:9200/_river/person/_meta -d '{
  "type": "jdbc",
  "jdbc": {
    "driver": "com.microsoft.sqlserver.jdbc.SQLServerDriver",
    "url": "jdbc:sqlserver://127.0.0.1:1433;databaseName=blogcontext",
    "user": "sa",
    "password": "password",
    "sql": "select person_id as _id, name from person",
    "poll": "30s"
  },
  "index": {
    "index": "library",
    "type": "person",
    "bulk_size": 500,
    "autocommit": true
  }
}'

curl -XPUT localhost:9200/_river/work/_meta -d '{
  "type": "jdbc",
  "jdbc": {
    "driver": "com.microsoft.sqlserver.jdbc.SQLServerDriver",
    "url": "jdbc:sqlserver://127.0.0.1:1433;databaseName=blogcontext",
    "user": "sa",
    "password": "password",
    "sql": "select person_id as _parent,name,genre,publisher from work",
    "poll": "30s"
  },
  "index": {
    "index": "library",
    "type": "work",
    "bulk_size": 500,
    "autocommit": true
  }
}'

日志文件

   [2014-01-14 07:10:35,488][ERROR][OneShotRiverMouth        ] bulk [1] error
    org.elasticsearch.ElasticSearchIllegalArgumentException: Can't specify parent if no parent field has been configured
        at org.elasticsearch.action.index.IndexRequest.process(IndexRequest.java:597)
        at org.elasticsearch.action.bulk.TransportBulkAction.executeBulk(TransportBulkAction.java:165)
        at org.elasticsearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:140)
        at org.elasticsearch.action.bulk.TransportBulkAction.doExecute(TransportBulkAction.java:63)
        at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:63)
        at org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:92)
        at org.elasticsearch.client.support.AbstractClient.bulk(AbstractClient.java:149)
        at org.elasticsearch.action.bulk.BulkProcessor.execute(BulkProcessor.java:283)
        at org.elasticsearch.action.bulk.BulkProcessor.access$400(BulkProcessor.java:46)
        at org.elasticsearch.action.bulk.BulkProcessor$Flush.run(BulkProcessor.java:336)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)

谢谢

1 个回答
  • 假设你的表看起来像:

    table1
    table_id| name| prop1|prop2|prop3
    
    child_table1
    child_id| table_id| child_prop1|child_prop2|child_prop3
    

    您需要选择主行ID并将其命名为"_id",即您的父ID,并将其命名为"_parent"

    curl -XPUT 'localhost:9200/_river/parent/_meta' -d '{
        "type" : "jdbc",
        "jdbc" : {
            "driver" : "com.mysql.jdbc.Driver",
            "url" : "jdbc:mysql://localhost:3306/test",
            "user" : "",
            "password" : "",
            "sql" : "select table_id as _id, name, prop1, prop2, prop3 from table1",
            "index" : "index1",
            "type" : "parent"
        }
    }'
    
    curl -XPUT 'localhost:9200/_river/child/_meta' -d '{
        "type" : "jdbc",
        "jdbc" : {
            "driver" : "com.mysql.jdbc.Driver",
            "url" : "jdbc:mysql://localhost:3306/test",
            "user" : "",
            "password" : "",
            "sql" : "select child_id as _id, table_id as _parent, child_prop1, child_prop2, child_prop3 from child_table1",
            "index" : "index1",
            "type" : "child"
        }
    }'
    

    并像你一样定义映射父/子,然后就完成了.您现在可以使用父/子查询来查询父/子数据.

    UPDATE: 我已经使用了您最新的映射并创建了一个示例数据库来导入数据.一切正常,我可以索引父/子没有任何错误.我正在使用ES 0.9.5,jdbc-river 2.2.2.

    2023-02-05 08:32 回答
撰写答案
今天,你开发时遇到什么问题呢?
立即提问
热门标签
PHP1.CN | 中国最专业的PHP中文社区 | PNG素材下载 | DevBox开发工具箱 | json解析格式化 |PHP资讯 | PHP教程 | 数据库技术 | 服务器技术 | 前端开发技术 | PHP框架 | 开发工具 | 在线工具
Copyright © 1998 - 2020 PHP1.CN. All Rights Reserved 京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有