weixin_39653442
weixin_39653442
2020-12-25 23:52

Trained intents ignore some sentences return default utterance

Issues

I have existing trained intent, that should return utterance. Problem is that some sentences are ignored, others are recognized. These sentences worked after creation of intent, later when more intents/domains were defined some sentences were ignored. Ignored sentences are part of only one intent so there are no duplicates for entire chatbot.

Articulate v.0.9.2/linux AWS machine

Example different results for same intent: image

该提问来源于开源项目:samtecspg/articulate

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

4条回答

  • weixin_39876592 weixin_39876592 4月前

    Can you give us the full /parse output for that utterance? It's likely you just need more training data or to drop one of the thresholds down.

    I know you know this, but Articulate isn't performing a lookup, it's training a model. As such there's no guarantee that exact matches have high enough confidence to proceed. Especially since that intent has several words only used once like skills and request... it may not be enough to train the agent that those are important words. whereas help has 4 utterances.

    点赞 评论 复制链接分享
  • weixin_39653442 weixin_39653442 4月前

    I added three more sentences to the intent, which resolved the issue for now. What I find weird and difficult to follow is what amount of sentences are enough to reach threshold of the intent. The behavior manifested after I added more new intents. Which for some reason should lower the confidence level of matching intent. If I continue to add more intents, and the confidence will be lowered again. Suddenly previously matching intents for sentences and trained use cases do not work. I can only discover this, when writing and testing out all possibilities.

    点赞 评论 复制链接分享
  • weixin_39876592 weixin_39876592 4月前

    hmmm, well we're going to try and help a little bit, by adding links to the /parse results so you can see the confidences. #219

    But I think what you are saying makes sense. There are a few different directions things can go. If you add more and more examples that are very similar to each other you will "overfit" the model to that sentence type. It will become too specific.

    Another thing that can happen is that everything is working great and you add more examples to intent A to make it match better, but those examples are similar in some way to Intent B and they make intent B perform worse...

    I'm sure you are tired of me saying we're working on things, but there are two glimmers of hope down this avenue of thought: Rasa is getting ready to release a phrase matcher, which we will surely incorporate into Articulate. https://github.com/RasaHQ/rasa_nlu/pull/822 but then we have intentions of adding some analytics in to help value the model after every training. Allowing you to publish the changes/rollback more easily. part of that will be automated testing across the agent. like #171

    点赞 评论 复制链接分享
  • weixin_39653442 weixin_39653442 4月前

    Thank you. I know that there are lot of issues, but I can see that project is on good track, with the speed of responses. The use case for articulate is very fitting for my needs. Even that project is still WIP, it still is enough to prepare POC or MVP for customer to show. Even with all the issues it is still better doing everything by hand with only "rasa" platform. I can not imaging writing that training.json files or preparing stories.md definition for all dialog flows for rasa platform by hand.

    点赞 评论 复制链接分享

相关推荐