Robot.txt File (What is Robot.txt File in Hindi), (SEO, Generator, Checker, Blogger, WordPress, Use)
Is your blog post not crawling and indexing the way you want? Do you know what a robots.txt file is? What are the benefits of Robots.txt File? How to add robot.txt file to Blogger and WordPress? Often new bloggers have a lot of problems due to not knowing about Robots.txt File properly.
According to NP Digital co-founder Nell Patel, robots.txt file is a small text file of posts from all websites available on the Internet. Robots.txt File is specially created for search engines and SEO.Robots.txt File helps a lot in ranking your post .But still many bloggers do not understand its value. You're not one of them! If yes, this is not good for your website.
If you want your website to rank well in the search engine like other bloggers, then you will get all the information related to Robots.txt File in this article of mine, what is Robots.txt File, why Robot.txt file is necessary for SEO.
This article is going to be very useful for you, so do not miss a single point of it, so let's first start that Robots.txt File Kya Hai ( Robots.txt File Meaning In Hindi)
What is a robot.txt file?
Robots.txt File is also called robots exclusion protocol. Robots.txt File is known as a text file. Robots.txt File Google's web robots tell what is crawlable in your post and what is not. Robots.txt File sends the command you give to the search engine.
There are many bloggers who do not yet know exactly what is a robots.txt file. The name of Robots.txt File itself may have guessed that it is an extension of Text File in which you can only write text.
In other words, if you want to show a page from your blog in the search engine or you do not want your page to appear in the search engine, then you should give all these things in the robots.txt file.
In this way, later when the search engine like Google, Bing or Yahoo o starts indexing your post, then the search engine will have a command from the Robot.txt File. IN which there will be a message that which things of that blog have to be indexed and which are not.
The biggest advantage of this will be that your blog post becomes SEO friendly. Due to which the search engine easily understands which part to index and which not to do.
Why are robots.txt file important for SEO?
Now you know what is Robots.txt File. Now let's understand why robots.txt file is necessary for SEO. Robots.txt File Technical plays a very important role in SEO. While doing SEO of your post, you should keep in mind that you must add Robots.txt File in it.
Because when you submit the sitemap of your post once in Google search console, Google's bots come to your post and if they do not find the robots.txt file there, then in that situation they start crawling and indexing the content of your entire post.
In this, even those content is indexed which you did not want them to be. This process does not have a good impact on SEO. Indexing content with Google's crawlers instruction is why robots.txt files are used.
What are the benefits of Robots.txt File?
Now you have understood why Robots.txt File is important, Robots.txt File is important for SEO. Now we know what are the benefits of Robots.txt File.
Important things related to robots.txt file
Now I tell you about some important things related to Robots.txt File which are used in Robots.txt File.
- Allow – When you allow something inside your text file, it means that you want to show that part in the search engine.
- Disallow – If you disallow any of your posts, it means that you do not want to show it in the search engine.
- User agent – With the help of this, you can show people coming to your blog what you want to show them. Such as the source of the image or where the file is saved etc.
- User agent Media Partners Google (user agent: Media Partners Google) – You can use this to hide your AdSense code from visitors. But let us tell you that if you disallow it, then Google AdSense will not be able to show ads on your blog or page at all. It is better to keep it allow.
How to Add Robots.txt File in Blogger?
Due to the new update in Blogger, the way to add Robots.txt File has changed, in this first you have to go to Settings and then allow Custom enabled robots.txt in Settings.
Whenever you want to make your blog SEO friendly, then Robot.txt File is used at that time. But most people leave it like that. But Blogger creates a best robot.txt file for your post.
But here let us tell you that if you have migrated your blog from BlogSpot to WordPress, then you must use this feature.
In this way, you can easily use Robot.txt File in your blogger with great ease. By the way, I would advise you that if you do not use this feature on Blogger, it will be better for you. And if your website is on WordPress then you must use this feature.
How to Add Robots.txt File in WordPress?
- If you use WordPress then you must know about Yoast SEO and you must have installed it.
- To add a robots.txt file, you go to Yoast SEO of your website and click on the tools and press the file editor button.
- Now copy and paste the sitemap of your website here and then press the Save Changing to Robots text button .Then your robots.txt file will be added to WordPress.
Robot.txt File Some other information
FAQ
Q: What is the function of a robot.txt file?
Ans: To give commands to search engines.
Q: Where is the robot.txt file in a website?
Ans: In the root folder of the website.
Q: Is it necessary to use a robot.txt file for a blog or website?
Ans: Yes.
Q: What happens if the robot.txt file is not used?
Ans: In such a situation, the search engine can index everything.
Q: Should I use robot.txt file or in WordPress?
Ans : In WordPress.