Python script to submit URLs to Bing search engine

This is a tutorial on creating a Python script to automatically submit several URLs to the Bing search engine.

Bing Webmaster Tools

Bing Webmaster Tools is a free Microsoft service that allows webmasters to add their sites to the Bing crawler, bingbot, so they show up in the search engine. Bing is the second most popular search engine.

Bing Webmaster Tools also helps to monitor and maintain a website’s presence by visiting the added sitemaps.

Setup your Bing Webmaster Tools account

Create and setup your Bing Webmaster Tools account.

You can submit your sitemap by going to Sitemaps on the left panel. You can also submit individual URLs by going to URL Submission on the left panel.

Under URL submission, when you click on the Submit URLs button, it opens a popup where you enter a list of URLs. We will automate this part by writing a Python program that will use the URL Submission API.

Create an API Key

On the top-right of the page, beside your profile picture, click on the gears icon. Under Settings, click on API access.

Under API access, select View API Key. You can generate your own API key there.

Do NOT give your API key to anyone.

Create and Populate urllist.txt

Create a file named urllist.txt and enter your website URLs, one per line. Our Python program will read this file, collect all the URLs, format them properly and bulk submit them to Bing. My urllist.txtl looks like this:

urllist.txt

https://aruljohn.com/
https://aruljohn.com/about/
https://aruljohn.com/blog/

Create your Python script.

Create a file called bing_submission.py and copy/paste it with this code:

bing_submission.py

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

"""
This Python script reads urllist.txt which contains several URLs, one per line and will submit all the URLs to Bing.
This script uses Bing's API Key and URL Submission APIs.
The documentation is found at these two webpages:
    (1) https://docs.microsoft.com/en-us/bingwebmaster/getting-access 
    (2) https://www.bing.com/webmasters/url-submission-api#APIs
Written by Arul John
Blog Post: https://aruljohn.com/blog/python-bing-submission-api/
"""

import requests

# Read all URLs in urllist.txt, join them and return the string
def get_urllist(urllist='urllist.txt'):
    urlstr = ''
    try:
        with open(urllist) as f:
            urls = f.readlines()
            urls = [f'"{u.strip()}"' for u in urls]
            urlstr = ', '.join(urls)
    except FileNotFoundError:
        print(f'ERROR: File {urllist} was not found')
        exit()
    except Exception:
        print('ERROR, quitting')
        exit()
    return f'{urlstr}'

# Constants - replace tehse values with your own
myurl = 'https://aruljohn.com'  # replace with your own URL
api_key = '1234567890'          # replace with your own API key

# Variables
url = f'https://ssl.bing.com/webmaster/api.svc/json/SubmitUrlbatch?apikey={api_key}'
headers = { 'Content-Type': 'application/json' }

# Main
if __name__ == '__main__':
    urlliststr = get_urllist()
    data = f'{{ "siteUrl":"{myurl}", "urlList":[{urlliststr}] }}'
    response = requests.post(url, headers=headers, data=data)
    print(response.status_code) # 200 is successful submission
    print(response.text)        # {"d":null} is a successful output

This Python program uses the requests module. So, if you do not have the requests module, please install it with:

pip install requests

Replace the value of the myurl variable with your own domain name.

Replace the value of api_key with your own API key.

Make sure that you have a file urllist.txt with one URL per line.

Run this program

When you run this program, it should return output similar to this:

$ python bing_submission.py 
200
{"d":null}

If the submission is successful, the first line of the output will contain 200 and the second line will show {"d":null}.

HTTP 400 error

If you get a 400 error instead of 200, please check your URL and API key. The URL has to be encoded and should not have any spaces. You may also get HTTP error 400 if the API key is wrong. You can always recreate a new API key.

Bing Webmaster Tools limits on URLs

Bing Webmaster Tools limits URL submissions to 10,000 a day per domain. This number or terms of condition can change any time, so please check the API documentation if you run into API restricting errors such as 429 (rate-limiting).

Conclusion

Were you able to submit your URLs to Bing with this program? If you have anything to report, please contact me.

Related Posts

If you have any questions, please contact me at arulbOsutkNiqlzziyties@gNqmaizl.bkcom. You can also post questions in our Facebook group. Thank you.