How to get HTML source of a Web Element in Selenium WebDriver

Related Product

Selenium WebDriver permit users to automate web browser for testing and data extraction tasks. When working with Selenium, obtaining the HTML germ of a specific web element is often necessary rather than the entire page.

Understanding how to extract an element & # 8217; s HTML is useful for debugging, verifying content, or further mechanisation steps. Read this article to learn how to get HTML origin of a web ingredient in WebDriver.

What is HTML Source?

This concern to the underlie a certain web element on a web page. Since it is the substructure of any web page, test HTML code in a normal browser and scenarios becomes vital. Although, do not confuse this with theHTML & lt; source & gt; tag.

What is a Web Element?

Anything that appear on a web page is a web element. Most obviously, this refers to text boxes, checkboxes, buttons, or any other fields that display or require data from the user. Web elements can also signify the tags within the web page & # 8217; s HTML codification. Essentially, interaction with the HTML code is interaction with a web element. Such elements usually experience unique identifiers, such as ID, gens, or unique course.

For example, to highlight text on a page, one would experience to interact with the “ body ”, a “ div ” and perhaps yet a “ p ” ingredient.

It is common for web constituent to occur within other web constituent. One can use mechanism such as or to situate them. You.

How to get HTML source of a web factor habituate Python?

To start with, download the Python bindings forSelenium WebDriver.

• One can do this from thePyPIpage for the Selenium bundle.
• Alternatively, one can usepipto install the Selenium packet.Python 3.6provide thepipin the standard library. Install Selenium withpipwith the following syntax:

pip install selenium

It is also possible to usevirtualenvto create isolated Python environments. Python 3.6 offerspyvenvwhich is quite alike tovirtualenv.

Notes for Windows users

1. Install Python 3.6with the MSI provided in the python.org download page.
2. Start a dictation prompt expend the cmd.exe program. Then run the pip command with the syntax given below to install Selenium.

C: Python35Scriptspip.exe install selenium

Now, here ’ s how to get a web factor:

elem = wd.find_element_by_css_selector (' # my-id ')

Here ’ s how to get the HTML root for the full page:

wd.page_source

How to Get HTML Page Source in Selenium WebDriver Using Python?

To get the HTML page source in Selenium WebDriver using Python, there are several method available such as:

• driver.page_source,
• driver.execute_script,
• and XPath querying.

Each method proffer a distinguishable approach for retrieving the page seed, depending on the tryout requirements or the element being accessed.

The below sections discuss these in detail, with examples from thebstackdemo.com site.

Get HTML Page Source utilise driver.page_source

The driver.page_source attribute allows retrieval of the entire HTML source of the current page as a string. This method is idealistic when the entire page source is needed, regardless of specific factor.

Syntax:

page_source = driver.page_source

This example regain the entire HTML content of bstackdemo.com using the driver.page_source dimension.

Example Code:

from selenium import webdriver # Initialize WebDriver driver = webdriver.Chrome () # Open bstackdemo.com driver.get (`` https: //bstackdemo.com/ '') # Get the page origin page_source = driver.page_source # Output the page source (truncated for brevity) print (page_source [:500]) # Print foremost 500 characters of the rootage # Close the browser driver.quit ()

Output:

& lt;! DOCTYPE html & gt; & lt; html xmlns= '' http: //www.w3.org/1999/xhtml '' lang= '' en '' & gt; & lt; head & gt; & lt; meta charset= '' utf-8 '' / & gt; & lt; meta name= '' viewport '' content= '' width=device-width, initial-scale=1, shrink-to-fit=no '' / & gt; & lt; meta name= '' description '' content= '' This is a sample app to showcase BrowserStack Automate '' / & gt; & lt; meta name= '' source '' content= '' BrowserStack '' / & gt; & lt; title & gt; BrowserStack Demo App & lt; /title & gt; & lt; link rel= '' cutoff icon '' href= '' favicon.ico '' / & gt; & lt; link href= '' css/bootstrap.min.css '' rel= '' stylesheet '' / & gt; & lt; link href= '' css/fontawesome.min.css '' rel= '' stylesheet '' / & gt; & lt; link href= '' cs

Get HTML Page Source using driver.execute_script

The driver.execute_script method allows executing JavaScript on the page. It can regain the entire HTML source by executing JavaScript code that returns the HTML content of the page.

Syntax:

page_source = driver.execute_script (`` retrovert document.documentElement.outerHTML; '')

This method is beneficial when the HTML source is required after JavaScript execution or dynamic contented load. Here, the JavaScript code document.documentElement.outerHTML retrieves the full HTML of the page.

Example Code:

from selenium import webdriver # Initialize WebDriver driver = webdriver.Chrome () # Open bstackdemo.com driver.get (`` https: //bstackdemo.com/ '') # Execute JavaScript to get page source page_source = driver.execute_script (`` return document.documentElement.outerHTML; '') # Output the page source (truncate for briefness) print (page_source [:500]) # Print foremost 500 quality of the source # Close the browser driver.quit ()

Output:

Pro tip: Tools like SUSA can handle this autonomously — upload your app and get results without writing a single test script.

& lt;! DOCTYPE html & gt; & lt; html xmlns= '' http: //www.w3.org/1999/xhtml '' lang= '' en '' & gt; & lt; head & gt; & lt; meta charset= '' utf-8 '' / & gt; & lt; meta name= '' viewport '' content= '' width=device-width, initial-scale=1, shrink-to-fit=no '' / & gt; & lt; meta name= '' description '' content= '' This is a sample app to showcase BrowserStack Automate '' / & gt; & lt; meta name= '' author '' content= '' BrowserStack '' / & gt; & lt; title & gt; BrowserStack Demo App & lt; /title & gt; & lt; link rel= '' cutoff icon '' href= '' favicon.ico '' / & gt; & lt; link href= '' css/bootstrap.min.css '' rel= '' stylesheet '' / & gt; & lt; link href= '' css/fontawesome.min.css '' rel= '' stylesheet '' / & gt; & lt; link href= '' cs

Get HTML Page Source Using XPath

can select specific elements on the page and regain their HTML substance. This method is helpful when only a specific section of the page, such as a exceptional div or component, demand to be captured.

Syntax:

element_html = driver.find_element_by_xpath (`` your_xpath_expression '') .get_attribute (`` outerHTML '')

Scenario:

In this example, an XPath expression is used to find the HTML of a specific factor on the page (div.container). This is utilitarian when you want to extract HTML for a specific element without retrieving the total page.

Example Code:

from selenium import webdriver # Initialize WebDriver driver = webdriver.Chrome () # Open bstackdemo.com driver.get (`` https: //bstackdemo.com/ '') # Example XPath to get the HTML of a specific factor element_html = driver.find_element_by_xpath (`` //div [@ class='container '] '') .get_attribute (`` outerHTML '') # Output the HTML of the element (truncated for brevity) print (element_html [:500]) # Print first 500 fibre of the factor 's HTML # Close the browser driver.quit ()

Output:

& lt; div & gt; & lt; header & gt; & lt; nav & gt; & lt; a href= '' / '' & gt; BrowserStack Demo App & lt; /a & gt; & lt; button type= '' button '' aria-controls= '' navbarNav '' aria-expanded= '' false '' aria-label= '' Toggle seafaring '' & gt; & lt; span & gt; & lt; /span & gt; & lt; /button & gt; & lt; div & gt; & lt; ul & gt; & lt; li & gt; & lt; a href= '' /home '' & gt; Home & lt; /a & gt;

How to regain the HTML root of a web ingredient using Selenium?

There are two main methods for retrieving the HTML source of a specific web element in Selenium:

Using the innerHTML attribute and the outerHTML dimension.

These methods allow elicit the HTML content of factor, depending on whether the element & # 8217; s substance or the full element (including the tag itself) is required.

Method 1: Get HTML Source in Selenium with innerHTML dimension

The innerHTML attribute find the HTML content inside the selected element, excluding the ingredient & # 8217; s tag itself. This method is practical when extracting the substance inside an element (e.g., the text, fry elements, etc.), but not the element & # 8217; s tag.

Syntax:

element_inner_html = driver.find_element_by_xpath (`` your_xpath_expression '') .get_attribute (`` innerHTML '')

Scenario:

In this example, the innerHTML attribute is used to extract the HTML message inside a div factor with the class name site-header from bstackdemo.com. This method will retrovert the content inside the head, without the & lt; div & gt; tag itself.

Example Code:

from selenium import webdriver # Initialize WebDriver driver = webdriver.Chrome () # Open bstackdemo.com driver.get (`` https: //bstackdemo.com/ '') # Get the internal HTML of the header section header_inner_html = driver.find_element_by_xpath (`` //div [@ class='site-header '] '') .get_attribute (`` innerHTML '') # Output the inner HTML content (truncated for brevity) mark (header_inner_html [:500]) # Print foremost 500 characters of the content inside the header # Close the browser driver.quit ()

Output:

& lt; nav & gt; & lt; a href= '' / '' & gt; BrowserStack Demo App & lt; /a & gt; & lt; button type= '' button '' aria-controls= '' navbarNav '' aria-expanded= '' false '' aria-label= '' Toggle navigation '' & gt; & lt; span & gt; & lt; /span & gt; & lt; /button & gt; & lt; div & gt; & lt; ul & gt; & lt; li & gt; & lt; a href= '' /home '' & gt; Home & lt; /a & gt;

Read the innerHTMLdimension to get the source of the factor & # 8217; s content.innerHTMLis a property of a DOM element whose value is the HTML between the opening tag and stop tag.

For example, theinnerHTMLproperty in the codification below channel the value “text”

& lt; p & gt; a text & lt; /p & gt;

This property can use to regain or dynamically insert content on a web page. However, if it is used to do anything beyond inserting uncomplicated schoolbook, some differences may occur in how it run across different browsers. It is a full practice to test your website across browser and device, try now.

Implement theinnerHTMLattribute to get the HTML beginning in Selenium with the next syntax:

Python:

element.get_attribute ('innerHTML ')Java:

elem.getAttribute (`` innerHTML '');

C#:

element.GetAttribute (`` innerHTML '');

Ruby:

element.attribute (`` innerHTML '')

JS:

element.getAttribute ('innerHTML ');

PHP:

$ elem.getAttribute ('innerHTML ');

Method 2: Get HTML Source in Selenium with outerHTML

The outerHTML attribute retrieves the entire HTML of the selected ingredient, including the component & # 8217; s tag itself. This method is useful when the full HTML of an element, including its tag, is required.

Syntax:

element_outer_html = driver.find_element_by_xpath (`` your_xpath_expression '') .get_attribute (`` outerHTML '')

Scenario:

In this exemplar, the outerHTML property is used to retrieve the accomplished HTML, including the div tag, of the div element with the stratum name site-header from bstackdemo.com. This is helpful when the full HTML construction of the element needs to be captured.

Example Code:

from selenium import webdriver # Initialize WebDriver driver = webdriver.Chrome () # Open bstackdemo.com driver.get (`` https: //bstackdemo.com/ '') # Get the outer HTML of the heading section header_outer_html = driver.find_element_by_xpath (`` //div [@ class='site-header '] '') .get_attribute (`` outerHTML '') # Output the outer HTML content (truncated for brevity) mark (header_outer_html [:500]) # Print first 500 characters of the element 's full HTML # Close the browser driver.quit ()

Output:

& lt; div & gt; & lt; nav & gt; & lt; a href= '' / '' & gt; BrowserStack Demo App & lt; /a & gt; & lt; button type= '' push '' aria-controls= '' navbarNav '' aria-expanded= '' false '' aria-label= '' Toggle navigation '' & gt; & lt; span & gt; & lt; /span & gt; & lt; /button & gt; & lt; div & gt; & lt; ul & gt; & lt; li & gt; & lt; a href= '' /home '' & gt; Home & lt; /a & gt;

Read the outerHTMLto get the origin with the current component.outerHTMLis an element property whose value is the HTML between the opening and closing tag and the HTML of the choose element itself.

For example, the code & # 8217; souterHTMLproperty pack a value that containsdiv and spaninside that.

& lt; div & gt; & lt; span & gt; Hello there! & lt; /span & gt; & lt; /div & gt;

Implement theouterHTMLattribute to get the HTML source in Selenium with the undermentioned syntax:

ele.get_atrribute (`` outerHTML '')

Automated selenium testing becomes more efficient and result-driven by apply the code detail supra. Detect, with ease, the HTML source of designated web component so that they may be examined for anomaly. Needless to say, identifying anomalies quickly leads to evenly quick debugging, thus pushing out site that provide optimum user experiences in minimum timelines.

Importance of Testing on Real Device Cloud with BrowserStack

Extracting a web element ’ s HTML origin is essential for debugging, contented check, and captivate dynamical information in automated examination. Selenium WebDriver get it easy to retrieve and formalise element details.

BrowserStack ’ s runs your Selenium exam on actual browsers and devices, not, providing accurate user experience perceptiveness. It volunteer seamless, with key advantages.

Why Test on Real Devices with BrowserStack Automate

• :Detect UI glitch and behavior matter that only appear on genuine devices.
• :Access thousands of real device and browser combinations to ensure broad compatibility.
• :Instantly run and scale tests without managing physical hardware.
• Remote Collaboration:Test anytime, anywhere, enable effective teamwork across distributed teams.

Talk to an Expert

Conclusion

Getting the HTML source of a web element in Selenium WebDriver is essential for validating page content and checking element structures.

Testing on a existent device cloud, such as BrowserStack, further ensures accurate event across different devices and browsers, direct to more true web applications.

Utile Resources for Selenium and Python

Related Guides