Was ist semrushbot?
indexiert und analysiert Inhalte von Webseiten.
Der Webrobot semrushbot indexiert und analysiert Inhalte von Webseiten. Er zeigt sich meistens mit der IP Adresse 104.131.42.50 und unter Verwendung des User Agent Mozilla/5.0 (compatible; SemrushBot; +http://www.semrush.com/bot.html). Mit 14.6% Marktanteil ist semrushbot auf Platz 3 der aktivsten Webrobots im Internet.
„SEMrush.com -> Bot"
— Offizielle Beschreibung des Betreibers
robots.txt – semrushbot blockieren
Füge diese Zeilen in deine robots.txt ein, um semrushbot den Zugriff auf deine Website zu verwehren:
robots.txt
User-agent: semrushbot
Disallow: /Du kannst den Zugriff auch gezielt einschränken, statt ihn komplett zu blockieren:
robots.txt (selektiv)
User-agent: semrushbot
Disallow: /wp-admin/
Disallow: /wp-includes/
Allow: /IP-Adressen 110 bekannte IPs
Diese IP-Adressen wurden bisher von semrushbot verwendet:
104.131.42.50
192.243.53.51
192.243.56.76
213.174.146.212
213.174.146.213
213.174.147.83
213.174.152.181
213.174.152.182
213.174.152.183
213.174.152.184
213.174.152.185
213.174.152.186
213.174.152.187
46.229.161.131
46.229.164.100
46.229.164.101
46.229.164.102
46.229.164.111
46.229.164.112
46.229.164.113
46.229.164.114
46.229.164.115
46.229.164.97
46.229.164.98
46.229.164.99
46.229.168.129
46.229.168.130
46.229.168.131
46.229.168.132
46.229.168.133
46.229.168.134
46.229.168.135
46.229.168.136
46.229.168.137
46.229.168.138
46.229.168.139
46.229.168.140
46.229.168.141
46.229.168.142
46.229.168.143
46.229.168.144
46.229.168.145
46.229.168.146
46.229.168.147
46.229.168.148
46.229.168.149
46.229.168.150
46.229.168.151
46.229.168.152
46.229.168.153
46.229.168.154
46.229.168.161
46.229.168.162
46.229.168.163
46.229.168.65
46.229.168.66
46.229.168.67
46.229.168.68
46.229.168.69
46.229.168.70
46.229.168.71
46.229.168.72
46.229.168.73
46.229.168.74
46.229.168.75
46.229.168.76
46.229.168.77
46.229.168.78
46.229.168.79
46.229.168.80
46.229.168.81
46.229.168.82
46.229.168.83
46.229.168.84
46.229.168.85
46.229.168.86
46.229.173.137
46.229.173.66
46.248.173.98
85.208.96.1
85.208.96.193
85.208.96.194
85.208.96.195
85.208.96.196
85.208.96.197
85.208.96.198
85.208.96.199
85.208.96.2
85.208.96.200
85.208.96.201
85.208.96.202
85.208.96.203
85.208.96.204
85.208.96.205
85.208.96.206
85.208.96.207
85.208.96.208
85.208.96.209
85.208.96.210
85.208.96.211
85.208.96.212
85.208.96.65
85.208.96.66
85.208.96.67
85.208.96.68
85.208.96.69
85.208.96.7
85.208.96.70
85.208.96.71
85.208.98.18
User Agents
Mit diesen User-Agent-Strings identifiziert sich semrushbot:
Mozilla/5.0 (compatible; SemrushBot; +http://www.semrush.com/bot.html)Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html)Mozilla/5.0 (compatible; SemrushBot/6~bl; http://www.semrush.com/bot.html)Mozilla/5.0 (compatible; SemrushBot/1.0~bm; http://www.semrush.com/bot.html)SEMrushBotMozilla/5.0 (compatible; SemrushBot/3~bl; http://www.semrush.com/bot.html)Mozilla/5.0 (compatible; SemrushBot-BA; http://www.semrush.com/bot.html)Mozilla/5.0 (compatible; SemrushBot/2~bl; http://www.semrush.com/bot.html)Mozilla/5.0 (compatible; SemrushBot-SA/0.97; http://www.semrush.com/bot.html)Mozilla/5.0 (compatible; SemrushBot-SI/0.97; http://www.semrush.com/bot.html)Mozilla/5.0 (compatible; SemrushBot/1.2~bl; http://www.semrush.com/bot.html)Mozilla/5.0 (compatible; SemrushBot/0.98~bl; http://www.semrush.com/bot.html)