weboob.browser.selenium

class weboob.browser.selenium.SeleniumBrowser(logger=None, proxy=None, responses_dirname=None, weboob=None, proxy_headers=None)

Bases: object

Browser similar to PagesBrowser, but using Selenium.

URLs instances can be used. The need_login decorator can be used too.

Differences: - since JS code can be run anytime, the current url and page can change anytime - it’s not possible to use open(), only location() can be used - many options are not implemented yet (like proxies) or cannot be implemented at all

BASEURL = None
DEFAULT_WAIT = 10

Default wait time for wait_* methods

DRIVER

alias of selenium.webdriver.firefox.webdriver.WebDriver

HEADLESS = True

Run without any display

MAX_SAVED_RESPONSES = 1073741824
WINDOW_SIZE = None

Rendering window size

It can be useful for responsive websites which show or hide elements depending on the viewport size.

absurl(uri, base=None)
clear_storage()

Clear local storage.

deinit()
export_session()
get_storage()

Get localStorage content for current domain.

As for cookies, this method only manipulates data for current domain. It’s not possible to get all localStorage content. To get localStorage for multiple domains, the browser must change the url to each domain and call get_storage each time after. To do so, it’s wise to choose a neutral URL (like an image file or JS file) to avoid the target page itself changing the cookies.

implicit_wait(**kwds)

Context manager to change implicit wait time and restore it

Example:

with browser.implicit_wait(10):
    # Within this block, the implicit wait will be set to 10 seconds
    # and be restored at the end of block.
    # If the link is not found immediately, it will be periodically
    # retried until found (for max 10 seconds).
    el = self.find_element_link_text("Show list")
    el.click()
implicitly_wait(timeout)

Set implicit wait time

When querying anything in DOM in Selenium, like evaluating XPath, if not found, Selenium will wait in a blocking manner until it is found or until the implicit wait timeouts. By default, it is 0, so if an XPath is not found, it fails immediately.

Parameters:timeout – new implicit wait time in seconds
in_frame(**kwds)

Context manager to execute a block inside a frame and restore main page after.

In selenium, to operate on a frame’s content, one needs to switch to the frame before and return to main page after.

Parameters:selector – selector to match the frame

Example:

with self.in_frame(xpath_locator('//frame[@id="foo"]')):
    el = self.find_element_by_xpath('//a[@id="bar"]')
    el.click()
location(url, data=None, headers=None, params=None, method=None, json=None)

Change current url of the browser.

Warning: unlike other requests-based weboob browsers, this function does not block until the page is loaded, it’s completely asynchronous. To use the new page content, it’s necessary to wait, either implicitly (e.g. with context manager implicit_wait) or explicitly (e.g. using method wait_until)

open(*args, **kwargs)
page
save_response()
save_response_if_changed()
update_storage(d)

Update local storage content for current domain.

It has the same restrictions as get_storage.

url
wait_until(condition, timeout=None)

Wait until some condition object is met

Wraps WebDriverWait. See https://seleniumhq.github.io/selenium/docs/api/py/webdriver_support/selenium.webdriver.support.wait.html

See CustomCondition.

Parameters:timeout – wait time in seconds (else DEFAULT_WAIT if None)
wait_until_is_here(urlobj, timeout=None)
wait_xpath(xpath, timeout=None)
wait_xpath_clickable(xpath, timeout=None)
wait_xpath_invisible(xpath, timeout=None)
wait_xpath_visible(xpath, timeout=None)
class weboob.browser.selenium.SeleniumPage(browser)

Bases: object

Page to use in a SeleniumBrowser

Differences with regular weboob Pages: - cannot access raw HTML text

doc
is_here()

Method to determine if the browser is on this page and the page is ready.

Use XPath and page content to determine if we are on this page. Make sure the page is “ready” for the usage we want. For example, if there’s a splash screen in front the page, preventing click, it should return False.

is_here can be a method or a CustomCondition instance.

logged = False
class weboob.browser.selenium.HTMLPage(browser)

Bases: weboob.browser.pages.HTMLPage

Accept any arguments, necessary for AbstractPage __new__ override.

AbstractPage, in its overridden __new__, removes itself from class hierarchy so its __new__ is called only once. In python 3, default (object) __new__ is then used for next instantiations but it’s a slot/”fixed” version supporting only one argument (type to instanciate).

ENCODING = u'utf-8'
class weboob.browser.selenium.CustomCondition

Bases: object

Abstract condition class

In Selenium, waiting is done on callable objects named “conditions”. Basically, a condition is a function predicate returning True if some condition is met.

The builtin selenium conditions are in selenium.webdriver.support.expected_conditions.

This class exists to differentiate normal methods from condition objects when calling SeleniumPage.is_here.

See https://seleniumhq.github.io/selenium/docs/api/py/webdriver_support/selenium.webdriver.support.expected_conditions.html When using selenium.webdriver.support.expected_conditions, it’s better to wrap them using WrapException.

class weboob.browser.selenium.AnyCondition(*conditions)

Bases: weboob.browser.selenium.CustomCondition

Condition that is true if any of several conditions is true.

class weboob.browser.selenium.AllCondition(*conditions)

Bases: weboob.browser.selenium.CustomCondition

Condition that is true if all of several conditions are true.

class weboob.browser.selenium.NotCondition(condition)

Bases: weboob.browser.selenium.CustomCondition

Condition that tests the inverse of another condition.

class weboob.browser.selenium.IsHereCondition(urlobj)

Bases: weboob.browser.selenium.CustomCondition

Condition that is true if a page “is here”.

This condition is to be passed to SeleniumBrowser.wait_until. It mustn’t be used in a SeleniumPage.is_here definition.

weboob.browser.selenium.VisibleXPath(xpath)

Wraps visibility_of_element_located

weboob.browser.selenium.ClickableXPath(xpath)

Wraps element_to_be_clickable

weboob.browser.selenium.ClickableLinkText(text, partial=False)

Wraps element_to_be_clickable

weboob.browser.selenium.HasTextCondition(xpath)

Condition to ensure some xpath is visible and contains non-empty text.

class weboob.browser.selenium.WrapException(condition)

Bases: weboob.browser.selenium.CustomCondition

Wrap Selenium’s builtin expected_conditions to catch exceptions.

Selenium’s builtin expected_conditions return True when a condition is met but might throw exceptions when it’s not met, which might not be desirable.

WrapException wraps such expected_conditions to catch those exception and simply return False when such exception is thrown.

weboob.browser.selenium.xpath_locator(xpath)

Creates an XPath locator from a string

Most Selenium functions don’t accept XPaths directly but “locators”. Locators can be XPath, CSS selectors.

Creates an link text locator locator from a string

Most Selenium functions don’t accept XPaths directly but “locators”.

Warning: if searched text is not directly in <a> but in one of its children, some webdrivers might not find the link.

class weboob.browser.selenium.ElementWrapper(wrapped)

Bases: object

Wrapper to Selenium element to ressemble lxml.

Some differences: - only a subset of lxml’s Element class are available - cannot access XPath “text()”, only Elements

See https://seleniumhq.github.io/selenium/docs/api/py/webdriver_remote/selenium.webdriver.remote.webelement.html

attrib
itertext()
text
text_content()
xpath(xpath)

Returns a list of elements matching xpath.

Since it uses find_elements_by_xpath, it does not raise NoSuchElementException or TimeoutException.