tld package

Submodules

tld.base module

class tld.base.BaseTLDSourceParser

Bases: object

Base TLD source parser.

classmethod get_tld_names(fail_silently: bool = False, retry_count: int = 0)

Get tld names.

Parameters

• fail_silently –

• retry_count –

Returns

include_private: bool = True

local_path: str

source_url: str

uid: Optional[str] = None

classmethod update_tld_names(fail_silently: bool = False) → bool

Update the local copy of the TLD file.

Parameters

fail_silently –

Returns

classmethod validate()

Constructor.

class tld.base.Registry(name, bases, attrs)

Bases: type

REGISTRY: Dict[str, tld.base.BaseTLDSourceParser] = {'mozilla': <class 'tld.utils.MozillaTLDSourceParser'>, 'mozilla_public_only': <class 'tld.utils.MozillaPublicOnlyTLDSourceParser'>}

classmethod get(key: str, default: Optional[tld.base.BaseTLDSourceParser] = None) → Optional[tld.base.BaseTLDSourceParser]

classmethod items() → ItemsView[str, tld.base.BaseTLDSourceParser]

classmethod reset() → None

tld.conf module

tld.conf.get_setting(name: str, default: Optional[Any] = None) → Any

Gets a variable from local settings.

Parameters

• name (str) –

• default (mixed) – Default value.

Return mixed

tld.conf.reset_settings() → None

Reset settings.

tld.conf.set_setting(name: str, value: Any) → None

Override default settings.

Parameters

• name (str) –

• value (mixed) –

tld.defaults module

tld.exceptions module

exception tld.exceptions.TldBadUrl(url)

Bases: ValueError

TldBadUrl.

Supposed to be thrown when bad URL is given.

exception tld.exceptions.TldDomainNotFound(domain_name)

Bases: ValueError

TldDomainNotFound.

Supposed to be thrown when domain name is not found (didn’t match) the local TLD policy.

exception tld.exceptions.TldIOError

Bases: OSError

TldIOError.

Supposed to be thrown when problems with reading/writing occur.

exception tld.exceptions.TldImproperlyConfigured

Bases: Exception

TldImproperlyConfigured.

Supposed to be thrown when code is improperly configured. Typical use-case is when user tries to use get_tld function with both search_public and search_private set to False.

tld.helpers module

tld.helpers.PROJECT_DIR(base: str) → str

Project dir.

tld.helpers.project_dir(base: str) → str

Project dir.

tld.registry module

class tld.registry.Registry(name, bases, attrs)

Bases: type

REGISTRY: Dict[str, tld.base.BaseTLDSourceParser] = {'mozilla': <class 'tld.utils.MozillaTLDSourceParser'>, 'mozilla_public_only': <class 'tld.utils.MozillaPublicOnlyTLDSourceParser'>}

classmethod get(key: str, default: Optional[tld.base.BaseTLDSourceParser] = None) → Optional[tld.base.BaseTLDSourceParser]

classmethod items() → ItemsView[str, tld.base.BaseTLDSourceParser]

classmethod reset() → None

tld.result module

class tld.result.Result(tld: str, domain: str, subdomain: str, parsed_url: urllib.parse.SplitResult)

Bases: object

Container.

domain

property extension

Alias of tld.

Return str

property fld

First level domain.

Returns

Return type

str

parsed_url

subdomain

property suffix

Alias of tld.

Return str

tld

tld.trie module

class tld.trie.Trie

Bases: object

An adhoc Trie data structure to store tlds in reverse notation order.

add(tld: str, private: bool = False) → None

class tld.trie.TrieNode

Bases: object

Class representing a single Trie node.

children

exception

leaf

private

tld.utils module

class tld.utils.BaseMozillaTLDSourceParser

Bases: tld.base.BaseTLDSourceParser

classmethod get_tld_names(fail_silently: bool = False, retry_count: int = 0) → Optional[Dict[str, tld.trie.Trie]]

Parse.

Parameters

• fail_silently –

• retry_count –

Returns

local_path: str

source_url: str

class tld.utils.MozillaPublicOnlyTLDSourceParser

Bases: tld.utils.BaseMozillaTLDSourceParser

Mozilla TLD source.

include_private: bool = False

local_path: str = 'res/effective_tld_names_public_only.dat.txt'

source_url: str = 'https://publicsuffix.org/list/public_suffix_list.dat?publiconly'

uid: str = 'mozilla_public_only'

class tld.utils.MozillaTLDSourceParser

Bases: tld.utils.BaseMozillaTLDSourceParser

Mozilla TLD source.

local_path: str = 'res/effective_tld_names.dat.txt'

source_url: str = 'https://publicsuffix.org/list/public_suffix_list.dat'

uid: str = 'mozilla'

class tld.utils.Result(tld: str, domain: str, subdomain: str, parsed_url: urllib.parse.SplitResult)

Bases: object

Container.

domain

property extension

Alias of tld.

Return str

property fld

First level domain.

Returns

Return type

str

parsed_url

subdomain

property suffix

Alias of tld.

Return str

tld

tld.utils.get_fld(url: Union[str, urllib.parse.SplitResult], fail_silently: bool = False, fix_protocol: bool = False, search_public: bool = True, search_private: bool = True, parser_class: Optional[Type[tld.base.BaseTLDSourceParser]] = None, **kwargs) → Optional[str]

Extract the first level domain.

Extract the top level domain based on the mozilla’s effective TLD names dat file. Returns a string. May throw TldBadUrl or TldDomainNotFound exceptions if there’s bad URL provided or no TLD match found respectively.

Parameters

• url (str | SplitResult) – URL to get top level domain from.

• fail_silently (bool) – If set to True, no exceptions are raised and None is returned on failure.

• fix_protocol (bool) – If set to True, missing or wrong protocol is ignored (https is appended instead).

• search_public (bool) – If set to True, search in public domains.

• search_private (bool) – If set to True, search in private domains.

• parser_class –

Returns

String with top level domain (if as_object argument is set to False) or a tld.utils.Result object (if as_object argument is set to True); returns None on failure.

Return type

str

tld.utils.get_tld(url: Union[str, urllib.parse.SplitResult], fail_silently: bool = False, as_object: bool = False, fix_protocol: bool = False, search_public: bool = True, search_private: bool = True, parser_class: Optional[Type[tld.base.BaseTLDSourceParser]] = None) → Optional[Union[str, tld.result.Result]]

Extract the top level domain.

Extract the top level domain based on the mozilla’s effective TLD names dat file. Returns a string. May throw TldBadUrl or TldDomainNotFound exceptions if there’s bad URL provided or no TLD match found respectively.

Parameters

• url (str | SplitResult) – URL to get top level domain from.

• fail_silently (bool) – If set to True, no exceptions are raised and None is returned on failure.

• as_object (bool) – If set to True, tld.utils.Result object is returned, domain, suffix and tld properties.

• fix_protocol (bool) – If set to True, missing or wrong protocol is ignored (https is appended instead).

• search_public (bool) – If set to True, search in public domains.

• search_private (bool) – If set to True, search in private domains.

• parser_class –

Returns

String with top level domain (if as_object argument is set to False) or a tld.utils.Result object (if as_object argument is set to True); returns None on failure.

Return type

str

tld.utils.get_tld_names(fail_silently: bool = False, retry_count: int = 0, parser_class: Optional[Type[tld.base.BaseTLDSourceParser]] = None) → Dict[str, tld.trie.Trie]

Build the tlds list if empty. Recursive.

Parameters

• fail_silently (bool) – If set to True, no exceptions are raised and None is returned on failure.

• retry_count (int) – If greater than 1, we raise an exception in order to avoid infinite loops.

• parser_class (BaseTLDSourceParser) –

Returns

List of TLD names

Return type

obj:tld.utils.Trie

tld.utils.get_tld_names_container() → Dict[str, tld.trie.Trie]

Get container of all tld names.

Returns

Rtype dict

tld.utils.is_tld(value: Union[str, urllib.parse.SplitResult], search_public: bool = True, search_private: bool = True, parser_class: Optional[Type[tld.base.BaseTLDSourceParser]] = None) → bool

Check if given URL is tld.

Parameters

• value (str) – URL to get top level domain from.

• search_public (bool) – If set to True, search in public domains.

• search_private (bool) – If set to True, search in private domains.

• parser_class –

Returns

Return type

bool

tld.utils.parse_tld(url: Union[str, urllib.parse.SplitResult], fail_silently: bool = False, fix_protocol: bool = False, search_public: bool = True, search_private: bool = True, parser_class: Optional[Type[tld.base.BaseTLDSourceParser]] = None) → Union[Tuple[None, None, None], Tuple[str, str, str]]

Parse TLD into parts.

Parameters

• url –

• fail_silently –

• fix_protocol –

• search_public –

• search_private –

• parser_class –

Returns

Tuple (tld, domain, subdomain)

Return type

tuple

tld.utils.pop_tld_names_container(tld_names_local_path: str) → None

Remove TLD names container item.

Parameters

tld_names_local_path –

Returns

tld.utils.process_url(url: Union[str, urllib.parse.SplitResult], fail_silently: bool = False, fix_protocol: bool = False, search_public: bool = True, search_private: bool = True, parser_class: Type[tld.base.BaseTLDSourceParser] = <class 'tld.utils.MozillaTLDSourceParser'>) → Union[Tuple[List[str], int, urllib.parse.SplitResult], Tuple[None, None, urllib.parse.SplitResult]]

Process URL.

Parameters

• parser_class –

• url –

• fail_silently –

• fix_protocol –

• search_public –

• search_private –

Returns

tld.utils.reset_tld_names(tld_names_local_path: Optional[str] = None) → None

Reset the tld_names to empty value.

If tld_names_local_path is given, removes specified entry from tld_names instead.

Parameters

tld_names_local_path (str) –

Returns

tld.utils.update_tld_names(fail_silently: bool = False, parser_uid: str = None) → bool

Update TLD names.

Parameters

• fail_silently –

• parser_uid –

Returns

tld.utils.update_tld_names_cli() → int

CLI wrapper for update_tld_names.

Since update_tld_names returns True on success, we need to negate the result to match CLI semantics.

tld.utils.update_tld_names_container(tld_names_local_path: str, trie_obj: tld.trie.Trie) → None

Update TLD Names container item.

Parameters

• tld_names_local_path –

• trie_obj –

Returns

Module contents

class tld.Result(tld: str, domain: str, subdomain: str, parsed_url: urllib.parse.SplitResult)

Bases: object

Container.

domain

property extension

Alias of tld.

Return str

property fld

First level domain.

Returns

Return type

str

parsed_url

subdomain

property suffix

Alias of tld.

Return str

tld

tld.get_fld(url: Union[str, urllib.parse.SplitResult], fail_silently: bool = False, fix_protocol: bool = False, search_public: bool = True, search_private: bool = True, parser_class: Optional[Type[tld.base.BaseTLDSourceParser]] = None, **kwargs) → Optional[str]

Extract the first level domain.

Extract the top level domain based on the mozilla’s effective TLD names dat file. Returns a string. May throw TldBadUrl or TldDomainNotFound exceptions if there’s bad URL provided or no TLD match found respectively.

Parameters

• url (str | SplitResult) – URL to get top level domain from.

• fail_silently (bool) – If set to True, no exceptions are raised and None is returned on failure.

• fix_protocol (bool) – If set to True, missing or wrong protocol is ignored (https is appended instead).

• search_public (bool) – If set to True, search in public domains.

• search_private (bool) – If set to True, search in private domains.

• parser_class –

Returns

String with top level domain (if as_object argument is set to False) or a tld.utils.Result object (if as_object argument is set to True); returns None on failure.

Return type

str

tld.get_tld(url: Union[str, urllib.parse.SplitResult], fail_silently: bool = False, as_object: bool = False, fix_protocol: bool = False, search_public: bool = True, search_private: bool = True, parser_class: Optional[Type[tld.base.BaseTLDSourceParser]] = None) → Optional[Union[str, tld.result.Result]]

Extract the top level domain.

Extract the top level domain based on the mozilla’s effective TLD names dat file. Returns a string. May throw TldBadUrl or TldDomainNotFound exceptions if there’s bad URL provided or no TLD match found respectively.

Parameters

• url (str | SplitResult) – URL to get top level domain from.

• fail_silently (bool) – If set to True, no exceptions are raised and None is returned on failure.

• as_object (bool) – If set to True, tld.utils.Result object is returned, domain, suffix and tld properties.

• fix_protocol (bool) – If set to True, missing or wrong protocol is ignored (https is appended instead).

• search_public (bool) – If set to True, search in public domains.

• search_private (bool) – If set to True, search in private domains.

• parser_class –

Returns

String with top level domain (if as_object argument is set to False) or a tld.utils.Result object (if as_object argument is set to True); returns None on failure.

Return type

str

tld.get_tld_names(fail_silently: bool = False, retry_count: int = 0, parser_class: Optional[Type[tld.base.BaseTLDSourceParser]] = None) → Dict[str, tld.trie.Trie]

Build the tlds list if empty. Recursive.

Parameters

• fail_silently (bool) – If set to True, no exceptions are raised and None is returned on failure.

• retry_count (int) – If greater than 1, we raise an exception in order to avoid infinite loops.

• parser_class (BaseTLDSourceParser) –

Returns

List of TLD names

Return type

obj:tld.utils.Trie

tld.is_tld(value: Union[str, urllib.parse.SplitResult], search_public: bool = True, search_private: bool = True, parser_class: Optional[Type[tld.base.BaseTLDSourceParser]] = None) → bool

Check if given URL is tld.

Parameters

• value (str) – URL to get top level domain from.

• search_public (bool) – If set to True, search in public domains.

• search_private (bool) – If set to True, search in private domains.

• parser_class –

Returns

Return type

bool

tld.parse_tld(url: Union[str, urllib.parse.SplitResult], fail_silently: bool = False, fix_protocol: bool = False, search_public: bool = True, search_private: bool = True, parser_class: Optional[Type[tld.base.BaseTLDSourceParser]] = None) → Union[Tuple[None, None, None], Tuple[str, str, str]]

Parse TLD into parts.

Parameters

• url –

• fail_silently –

• fix_protocol –

• search_public –

• search_private –

• parser_class –

Returns

Tuple (tld, domain, subdomain)

Return type

tuple

tld.update_tld_names(fail_silently: bool = False, parser_uid: str = None) → bool

Update TLD names.

Parameters

• fail_silently –

• parser_uid –

Returns