WP REST APIを使ってWordPressの記事をElasticSearchから検索してみる

この記事はWordPress Advent Calendar 2015の１７日目に投稿すべきだった記事です。遅くなってほんますみません。 ElasticSearchでWordPressの記事を検索する方法の１つとして、 […]

この記事はWordPress Advent Calendar 2015の１７日目に投稿すべきだった記事です。

遅くなってほんますみません。

ElasticSearchでWordPressの記事を検索する方法の１つとして、WP REST APIを間に挟むパターンを試してみました。

ElasticSearchのセットアップなど

https://wp-kyoto.cdn.rabify.me/try-to-use-elasticsearch/
（この記事の内容は実運用するもので真似しちゃいかんですよ？）

WP REST APIから記事データのJSONを取得する

連携させるコード書くのめんどくさいので、とりあえず一旦JSONファイルをダウンロードしちゃいましょう。

% wget -O sample.json https://rest-api.dev/wp-json/wp/v2/posts      
--2015-12-20 14:56:15--  https://rest-api.dev/wp-json/wp/v2/posts
Resolving rest-api.dev... 192.168.33.10
Connecting to rest-api.dev|192.168.33.10|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/json]
Saving to: 'sample.json'

sample.json                                            [ <=>                                                                                                               ]  39.13K  --.-KB/s   in 0.001s 

2015-12-20 14:56:16 (37.0 MB/s) - 'sample.json' saved [40068]

ElasticSearchにインポートしてみる

bulk APIを使えば一発インポートできるみたいなので、やってみます。

% curl -XPUT 'https://search-api-test-XXX.ap-northeast-1.es.amazonaws.com/wordpress/post/_bulk' --data-binary "@sample.json"
{"error":"ActionRequestValidationException[Validation Failed: 1: no requests added;]","status":400}%

エラーが出ました。

実はBulk APIでインポートする際は以下のようにJSONを加工しないとエラーになります。

{"index":{"_id":"1"}}
{"account_number":1,"balance":39225,"firstname":"Amber","lastname":"Duke","age":32,"gender":"M","address":"880 Holmes Lane","employer":"Pyrami","email":"[email protected]","city":"Brogan","state":"IL"}

JSONを加工せずに入れるには、１件ずつデータを入れる必要があります。

と、いうことで「WP REST APIからの記事取得→ElasticSearchへのデータ追加」までを行うシェルを書いてみました。
https://gist.github.com/hideokamoto/acf434562e0ce8d5562b

やっつけ感ありますが、とりあえず動いたので追い追い手直しします。

インポートしたデータを検索する

では早速検索してみましょう。

ひとまず投入されたデータを取得します。（見やすくするため、jqで整形してます）

% curl -XPOST https://search-api-test-XXX.ap-northeast-1.es.amazonaws.com/wp/_search  | jq ".hits.hits[]._source.title.rendered" 
"Markup: HTML Tags and Formatting"
"Markup: Title With Markup"
"Template: More Tag"
"Markup: Title With Special Characters"
"Template: Featured Image (Vertical)"
"Template: Featured Image (Horizontal)"
"Hello world!"
"Template: Excerpt (Defined)"
"Markup: Text Alignment"
"Markup: Image Alignment"

件数と取得データを指定する

続いて２記事のタイトルだけ抜き出してみます。
POSTで以下の値を投げることで取得可能です。

{
  "fields":"title.rendered",
  "size"  :2
}

こちらが検索結果。

% curl -XPOST https://search-wp-api-6lzkfk24hbdy73uicw2ogzbygi.ap-northeast-1.es.amazonaws.com/wp/_search -d '{"size":2,"fields":"title.rendered"}' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   396  100   360  100    36   1758    175 --:--:-- --:--:-- --:--:--  1764
{
  "took": 6,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 10,
    "max_score": 1,
    "hits": [
      {
        "_index": "wp",
        "_type": "posts",
        "_id": "1178",
        "_score": 1,
        "fields": {
          "title.rendered": [
            "Markup: HTML Tags and Formatting"
          ]
        }
      },
      {
        "_index": "wp",
        "_type": "posts",
        "_id": "1173",
        "_score": 1,
        "fields": {
          "title.rendered": [
            "Markup: Title With Markup"
          ]
        }
      }
    ]
  }
}

あいまい検索にかける

再びfuzzy_like_thisであいまい検索をかけてみます。
POSTするクエリは以下のとおりです。

{
  "query":{
    "fuzzy_like_this":{
      "fields":["title.rendered","content.rendered"],
      "like_text":"mrkp"
    }
  }
}

実際にPOSTしてみて、意図した記事が取れてるかjqで整形して確認します。

% curl -XPOST https://search-wp-api-6lzkfk24hbdy73uicw2ogzbygi.ap-northeast-1.es.amazonaws.com/wp/_search -d  '{
"query":{"fuzzy_like_this":{"fields":["title.rendered","content.rendered"],"like_text":"mrkp"}}}' |jq ".hits.hits[]._source.title.rendered"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 31476  100 31378  100    98  17747     55  0:00:01  0:00:01 --:--:-- 17747
"Markup: Title With Special Characters"
"Markup: Title With Markup"
"Markup: HTML Tags and Formatting"
"Markup: Text Alignment"
"Markup: Image Alignment"

せっかくなので１記事だけ取得して、どんなJSONが取れるかみてみましょう。

% curl -XPOST https://search-wp-api-6lzkfk24hbdy73uicw2ogzbygi.ap-northeast-1.es.amazonaws.com/wp/_search -d  '{
"query":{"fuzzy_like_this":{"fields":["title.rendered","content.rendered"],"like_text":"mrkp"}},"size":1}' | jq .
{
  "took": 13,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 5,
    "max_score": 0.614891,
    "hits": [
      {
        "_index": "wp",
        "_type": "posts",
        "_id": "1174",
        "_score": 0.614891,
        "_source": {
          "id": 1174,
          "date": "2013-01-05T11:00:20",
          "date_gmt": "2013-01-05T18:00:20",
          "guid": {
            "rendered": "https://wptest.io/demo/?p=867"
          },
          "modified": "2013-01-05T11:00:20",
          "modified_gmt": "2013-01-05T18:00:20",
          "slug": "title-with-special-characters",
          "type": "post",
          "link": "https://rest-api.dev/archives/1174",
          "title": {
            "rendered": "Markup: Title With Special Characters"
          },
          "content": {
            "rendered": "<p>Putting special characters in the title should have no adverse effect on the layout or functionality.</p>\n<p>Special characters in the post title have been known to cause issues with JavaScript when it is minified, especially in the admin when editing the post itself (ie. issues with metaboxes, media upload, etc.).</p>\n<h2>Latin Character Tests</h2>\n<p>This is a test to see if the fonts used in this theme support basic Latin characters.</p>\n<table>\n<tbody>\n<tr>\n<td>!</td>\n<td>&#8220;</td>\n<td>#</td>\n<td>$</td>\n<td>%</td>\n<td>&amp;</td>\n<td>&#8216;</td>\n<td>(</td>\n<td>)</td>\n<td>*</td>\n</tr>\n<tr>\n<td>+</td>\n<td>,</td>\n<td>&#8211;</td>\n<td>.</td>\n<td>/</td>\n<td>0</td>\n<td>1</td>\n<td>2</td>\n<td>3</td>\n<td>4</td>\n</tr>\n<tr>\n<td>5</td>\n<td>6</td>\n<td>7</td>\n<td>8</td>\n<td>9</td>\n<td>:</td>\n<td>;</td>\n<td>&gt;</td>\n<td>=</td>\n<td>&lt;</td>\n</tr>\n<tr>\n<td>?</td>\n<td>@</td>\n<td>A</td>\n<td>B</td>\n<td>C</td>\n<td>D</td>\n<td>E</td>\n<td>F</td>\n<td>G</td>\n<td>H</td>\n</tr>\n<tr>\n<td>I</td>\n<td>J</td>\n<td>K</td>\n<td>L</td>\n<td>M</td>\n<td>N</td>\n<td>O</td>\n<td>P</td>\n<td>Q</td>\n<td>R</td>\n</tr>\n<tr>\n<td>S</td>\n<td>T</td>\n<td>U</td>\n<td>V</td>\n<td>W</td>\n<td>X</td>\n<td>Y</td>\n<td>Z</td>\n<td>[</td>\n<td></td>\n</tr>\n<tr>\n<td>]</td>\n<td>^</td>\n<td>_</td>\n<td>`</td>\n<td>a</td>\n<td>b</td>\n<td>c</td>\n<td>d</td>\n<td>e</td>\n<td>f</td>\n</tr>\n<tr>\n<td>g</td>\n<td>h</td>\n<td>i</td>\n<td>j</td>\n<td>k</td>\n<td>l</td>\n<td>m</td>\n<td>n</td>\n<td>o</td>\n<td>p</td>\n</tr>\n<tr>\n<td>q</td>\n<td>r</td>\n<td>s</td>\n<td>t</td>\n<td>u</td>\n<td>v</td>\n<td>w</td>\n<td>x</td>\n<td>y</td>\n<td>z</td>\n</tr>\n<tr>\n<td>{</td>\n<td>|</td>\n<td>}</td>\n<td>~</td>\n<td></td>\n<td></td>\n<td></td>\n<td></td>\n<td></td>\n<td></td>\n</tr>\n</tbody>\n</table>\n"
          },
          "excerpt": {
            "rendered": "<p>Putting special characters in the title should have no adverse effect on the layout or functionality. Special characters in the post title have been known to cause issues with JavaScript when it is minified, especially in the admin when editing the post itself (ie. issues with metaboxes, media upload, etc.). Latin Character Tests This is a test to see if the fonts used in this theme support basic Latin characters. ! &#8220; # $ % &amp; &#8216; ( ) * + , &#8211; . / 0 1 2 3 4 5 6 7 8 9 : ; &gt; = &lt; ? @ A B C D E F G H &hellip; <a href=\"https://rest-api.dev/archives/1174\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Markup: Title With Special Characters</span></a></p>\n"
          },
          "author": 2,
          "featured_image": 0,
          "comment_status": "closed",
          "ping_status": "closed",
          "sticky": false,
          "format": "standard",
          "_links": {
            "self": [
              {
                "href": "https://rest-api.dev/wp-json/wp/v2/posts/1174"
              }
            ],
            "collection": [
              {
                "href": "https://rest-api.dev/wp-json/wp/v2/posts"
              }
            ],
            "about": [
              {
                "href": "https://rest-api.dev/wp-json/wp/v2/types/post"
              }
            ],
            "author": [
              {
                "embeddable": true,
                "href": "https://rest-api.dev/wp-json/wp/v2/users/2"
              }
            ],
            "replies": [
              {
                "embeddable": true,
                "href": "https://rest-api.dev/wp-json/wp/v2/comments?post=1174"
              }
            ],
            "version-history": [
              {
                "href": "https://rest-api.dev/wp-json/wp/v2/posts/1174/revisions"
              }
            ],
            "https://api.w.org/attachment": [
              {
                "href": "https://rest-api.dev/wp-json/wp/v2/media?parent=1174"
              }
            ],
            "https://api.w.org/term": [
              {
                "taxonomy": "category",
                "embeddable": true,
                "href": "https://rest-api.dev/wp-json/wp/v2/posts/1174/categories"
              },
              {
                "taxonomy": "post_tag",
                "embeddable": true,
                "href": "https://rest-api.dev/wp-json/wp/v2/posts/1174/tags"
              },
              {
                "taxonomy": "post_format",
                "embeddable": true,
                "href": "https://rest-api.dev/wp-json/wp/v2/posts/1174/post_format"
              }
            ],
            "https://api.w.org/meta": [
              {
                "embeddable": true,
                "href": "https://rest-api.dev/wp-json/wp/v2/posts/1174/meta"
              }
            ]
          }
        }
      }
    ]
  }
}

まとめ

WordPress側ではWP REST APIでいろんなデータをJSON出力が可能です。
そしてElasticSearchでは（ちょっとめんどくさかったですが）JSONをPUT/POST/UPDATEすることで検索が可能になります。
これ、上手く使えばかなり強力な検索エンジン作れそうですね。

ということで遅くなった分だいぶマニアックな内容にしてみました。