Answering questions about an image using outside knowledge
Answering questions about an image using outside knowledge
KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQA
arXiv paper abstract https://arxiv.org/abs/2012.11014
arXiv PDF paper https://arxiv.org/pdf/2012.11014.pdf
GitHub https://github.com/facebookresearch/mmf/tree/master/projects/krisp
Facebook Research https://research.fb.com/publications/krisp-integrating-implicit-and-symbolic-knowledge-for-open-domain-knowledge-based-vqa/
One of the most challenging question types in VQA is when answering the question requires outside knowledge not present in the image.
… We tap into two types of knowledge representations and reasoning.
First, implicit knowledge which can be learned effectively from unsupervised language pre-training and supervised training …
Second, explicit, symbolic knowledge encoded in knowledge bases.
… We combine diverse sources of knowledge to cover the wide variety of knowledge needed to solve knowledge-based questions.
… KRISP (Knowledge Reasoning with Implicit and Symbolic rePresentations), significantly outperforms state-of-the-art on OK-VQA, the largest available dataset for open-domain knowledge-based VQA. …
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
LinkedIn https://www.linkedin.com/in/morris-lee-47877b7b
