> My robots.txt are not workking?

My robots.txt are not workking?

Posted at: 2015-03-04 
and they look like this, all commands are in one line

http://www.kshayaiklhd.tk/robots.txt

instead of robots.txt of http://www.google.com/robots.txt

Deleting and re-creating the file won't fix the issue.

Any web browser will render the content in one single line because it considers the content received as HTML instead of mere TEXT―due to the CONTENT-TYPE value in the HTTP response header that is set by default to HTML as shown below in the copy of the HTTP header of the request to your robots.txt file (last line):

HTTP/1.1 200 OK

Date: Fri, 18 Jan 2013 21:04:45 GMT

Server: Apache

X-Powered-By: PHP/5.2.17

Content-Length: 148

Connection: close

Content-Type: text/html

To fix the issue, header should instead contain "Content-Type: text/plain" as in Google HTTP response header. To do so, you can explicitly set the MIME type associated to the .txt extension by adding the following line in your .htaccess file:

AddType text/plain .txt

If it still doesn't work, the configuration has to be done in the administrative interface your ISP provides you with to manage your website―look for a section talking about "MIME Types". If you're lost, contact your ISP, they'd be able to do it for your at no charge.

―Pascal Perry

Online Findability and SEO Strategist

Delete current robots.txt file ...

create new robots.txt file -

User-agent: *

Disallow:

Disallow: /cgi-bin/

delete this file andgenerate new robots.txt file and upload it in servers.

Create a new one by following your competitors

and they look like this, all commands are in one line

http://www.kshayaiklhd.tk/robots.txt

instead of robots.txt of http://www.google.com/robots.txt