Training Computer Vision Transformers without Natural Images
--
Can Vision Transformers Learn without Natural Images?
Deep AI https://deepai.org/publication/can-vision-transformers-learn-without-natural-images
They pre-train Vision Transformers without any image collections and annotation labor. They experimentally verify it partially outperforms Self-Supervised Learning without using any natural images in the pre-training phase. It can interpret natural image datasets to a large extent. For example, the performance rates on the CIFAR-10 dataset are as follows: their proposal 97.6% vs. SimCLRv2 97.4% vs. ImageNet 98.0%
arXiv paper abstract https://arxiv.org/abs/2103.13023
arXiv PDF paper https://arxiv.org/pdf/2103.13023v1.pdf
Unlimited computer fractals can help train AI to see
Large datasets like ImageNet have supercharged the last 10 years of AI vision, but they are hard to produce and contain bias. Computer generated datasets provide an alternative.
M.I.T. Technology Review https://www.technologyreview.com/2021/02/04/1017486
Web site https://hirokatsukataoka16.github.io/Vision-Transformers-without-Natural-Images
GitHub https://github.com/hirokatsukataoka16/FractalDB-Pretrained-ResNet-PyTorch
Oral presentation http://hirokatsukataoka.net/pdf/accv20_kataoka_oral.pdf
Poster http://hirokatsukataoka.net/pdf/accv20_kataoka_poster.pdf
Stay up to date. Subscribe to my posts https://morrislee1234.wixsite.com/website/contact
Web site with my other posts by category https://morrislee1234.wixsite.com/website
LinkedIn https://www.linkedin.com/in/morris-lee-47877b7b
