2020-11-30 02:48

to_indexed_json cannot use `super`, limits :as mappings without hacks

Perhaps the main surprising thing to highlight: defining your own to_indexed_json means that using the :as option for mappings will be ineffective.

The inability to make use of super results from Tire::Model::Search's efforts not to conflict with existing methods when included -- Tire won't define it's default implementation if you've defined the method yourself, thus super can't work and you must instead resort to hacks like alias_method_chain to let Tire do its work of applying the :as options.

I haven't come up with any ideas yet for how to improve the situation, but thought I'd show how I came to realize this, to lend a real use case for discussion. Here's an example from a Rails app, including workaround:

module UserIndexing
  extend ActiveSupport::Concern

  included do
    include Tire::Model::Search
    include Tire::Model::Callbacks

    tire.mapping do
      indexes :id,          :index    => :not_analyzed
      indexes :name,        :analyzer => :snowball,    :boost => 50
      indexes :bio,         :analyzer => :snowball
      indexes :email,       :index    => :not_analyzed
      indexes :created_at,  :type     => :date
      indexes :updated_at,  :type     => :date
      indexes :roles,       :type     => :integer,
                            :as       => proc { read_attribute(:roles) }

    alias_method_chain :to_indexed_json, :overrides

  # Enable finding users by searching for the names of related companies.
  def to_indexed_json_with_overrides
    base = JSON.parse(to_indexed_json_without_overrides)
    overrides = serializable_hash(
      except: [:preferences, :roles],
      include: {
        relationships: {
          only: [:role],
          include: {
            company: { only: [:name] },

As an implementation detail, roles are implemented with a bitmask (storing an integer value in the database), but .roles is an overridden accessor that returns an Array. For the sake of supporting authorization code, we want that integer value in the elasticsearch index, hence the read_attribute.

Now at the point we've gone to all this effort, we could probably just merge in the read_attribute result in our to_indexed_json instead of using the :as option for this, but the above code evolved over time, and here we are.

I've seen your (quite sensible) explanation of the intended separation of uses for mapping vs. to_indexed_json. Based on that, for this case, it seems like using :as is really the wrong thing to do. Also, having read that, it becomes clear why you've chosen the examples that you have in the RDoc.

So, not a big problem here, I've worked my way through it as you can see. I guess my takeaway is just that it'd be nice to see a little bit of guidance in docs that :as is best applied for "computed field" sort of use cases, usually not formatting of an existing DB field. And there's still the matter of the surprising fact that defining to_indexed_json breaks :as -- other cases may certainly arise that are similar to my example above, but :as is desired for a field that really should be computed.

I'll take a crack at a doc patch and see what you think in a PR. Not sure how to approach the :as/super problem yet :smile:


  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答


  • weixin_39924481 weixin_39924481 5月前

    Hmm, from your explanation on SO I didn't get the "last resort" feeling about to_indexed_json ("they serve two different purposes"). With the "computed field" comment, I meant that you've chosen examples like this:

    # In Tire::Model::Indexing#mapping RDoc
    indexes :words, :as => 'content.split(/\W/).length'
    # In README
    indexes :content_size, :as => 'content.size'

    ... creating synthesized fields in the elasticsearch index that don't map to actual fields in the source database. That's the "aha" I described about choice of examples after reading the explanation: these seemed consistent with the usage you were suggesting on SO.

    Alternatively in my example above, I suppose I could achieve the same thing by eliminating to_indexed_json and making more elaborate use of :as options (or nested indexes blocks?). That seems more troublesome than taking advantage of to_json with includes, in the case of ActiveRecord. If that is preferable though, perhaps to_indexed_json should be heavily deemphasized in docs?

    As an aside, particularly if to_indexed_json is considered a last resort, I'm definitely even more fond of the recent pull request that added support for giving a Symbol to :as and having that dispatched to a method.

    点赞 评论 复制链接分享
  • weixin_39528000 weixin_39528000 5月前

    Hello, sorry for the delay.

    If that is preferable though, perhaps to_indexed_json should be heavily deemphasized in docs?

    Yes. The mapping capabilities have evolved over time, and I haven't made the changes neccessary in the docs. Over-using to_indexed_json was one of the worse decisions I did in the course of writing and maintaining the library.

    Going with mapping is definitely a better way how to define the serialization. to_indexed_json is the "close to the metal" thing.

    Should we leave the issue open, I think we need to reframe it a bit...

    点赞 评论 复制链接分享
  • weixin_39924481 weixin_39924481 5月前

    Hey , I'm on vacation and getting away from the keyboard a lot until after Christmas, so may be slow to give feedback, but if you start drafting some new docs in this area, or you want a sounding board for ideas on IRC or something, I'd be happy to lend fresh, novice eyes to them.

    点赞 评论 复制链接分享
  • weixin_39528000 weixin_39528000 5月前

    Any fresh ideas about the issue?

    点赞 评论 复制链接分享
  • weixin_39528000 weixin_39528000 5月前


    点赞 评论 复制链接分享
  • weixin_39924481 weixin_39924481 5月前

    Yes. The mapping capabilities have evolved over time, and I haven't made the changes neccessary in the docs.

    Are there any more current/elaborate examples you can point out, in integration tests or elsewhere, that I might get an understanding of how to better approach using it in favor of to_indexed_json? More insight on why the overuse proved to be a bad thing, to inform possible doc improvements?

    点赞 评论 复制链接分享