Extrainfo Descriptor
********************

Parsing for Tor extra-info descriptors. These are published by relays
whenever their server descriptor is published and have a similar
format. However, unlike server descriptors these don’t contain
information that Tor clients require to function and as such aren’t
fetched by default.

Defined in section 2.1.2 of the dir-spec, extra-info descriptors
contain interesting but non-vital information such as usage
statistics. Tor clients cannot request these documents for bridges.

Extra-info descriptors are available from a few sources…

* If you have ‘DownloadExtraInfo 1’ in your torrc…

   * control port via ‘GETINFO extra-info/digest/*’ queries

   * the ‘cached-extrainfo’ file in tor’s data directory

* Archived descriptors provided by CollecTor.

* Directory authorities and mirrors via their DirPort.

**Module Overview:**

   ExtraInfoDescriptor - Tor extra-info descriptor.
     |- RelayExtraInfoDescriptor - Extra-info descriptor for a relay.
     |- BridgeExtraInfoDescriptor - Extra-info descriptor for a bridge.
     |
     +- digest - calculates the upper-case hex digest value for our content

stem.descriptor.extrainfo_descriptor.DirResponse(enum)

   Enumeration for known statuses for ExtraInfoDescriptor’s
   dir_*_responses.

   +---------------------+--------------------------------------------------------+
   | DirResponse         | Description                                            |
   +=====================+========================================================+
   | **OK**              | network status requests that were answered             |
   +---------------------+--------------------------------------------------------+
   | **NOT_ENOUGH_SIGS** | network status wasn’t signed by enough authorities     |
   +---------------------+--------------------------------------------------------+
   | **UNAVAILABLE**     | requested network status was unavailable               |
   +---------------------+--------------------------------------------------------+
   | **NOT_FOUND**       | requested network status was not found                 |
   +---------------------+--------------------------------------------------------+
   | **NOT_MODIFIED**    | network status unmodified since If-Modified-Since time |
   +---------------------+--------------------------------------------------------+
   | **BUSY**            | directory was busy                                     |
   +---------------------+--------------------------------------------------------+

stem.descriptor.extrainfo_descriptor.DirStat(enum)

   Enumeration for known stats for ExtraInfoDescriptor’s
   dir_*_direct_dl and dir_*_tunneled_dl.

   +-----------------------+---------------------------------------------------------------+
   | DirStat               | Description                                                   |
   +=======================+===============================================================+
   | **COMPLETE**          | requests that completed successfully                          |
   +-----------------------+---------------------------------------------------------------+
   | **TIMEOUT**           | requests that didn’t complete within a ten minute timeout     |
   +-----------------------+---------------------------------------------------------------+
   | **RUNNING**           | requests still in process when measurement’s taken            |
   +-----------------------+---------------------------------------------------------------+
   | **MIN**               | smallest rate at which a descriptor was downloaded in B/s     |
   +-----------------------+---------------------------------------------------------------+
   | **MAX**               | largest rate at which a descriptor was downloaded in B/s      |
   +-----------------------+---------------------------------------------------------------+
   | **D1-4** and **D6-9** | rate of the slowest x/10 download rates in B/s                |
   +-----------------------+---------------------------------------------------------------+
   | **Q1** and **Q3**     | rate of the slowest and fastest quarter download rates in B/s |
   +-----------------------+---------------------------------------------------------------+
   | **MD**                | median download rate in B/s                                   |
   +-----------------------+---------------------------------------------------------------+

class stem.descriptor.extrainfo_descriptor.ExtraInfoDescriptor(raw_contents, validate=False)

   Bases: "stem.descriptor.Descriptor"

   Extra-info descriptor document.

   Variables:
      * **nickname** (*str*) – ***** relay’s nickname

      * **fingerprint** (*str*) – ***** identity key fingerprint

      * **published** (*datetime*) – ***** time in UTC when this
        descriptor was made

      * **geoip_db_digest** (*str*) – sha1 of the geoIP database
        file for IPv4 addresses

      * **geoip6_db_digest** (*str*) – sha1 of the geoIP database
        file for IPv6 addresses

      * **transport** (*dict*) – ***** mapping of transport methods
        to their (address, port, args) tuple, these usually appear on
        bridges in which case all of those are **None**

   **Bi-directional connection usage:**

   Variables:
      * **conn_bi_direct_end** (*datetime*) – end of the sampling
        interval

      * **conn_bi_direct_interval** (*int*) – seconds per interval

      * **conn_bi_direct_below** (*int*) – connections that
        read/wrote less than 20 KiB

      * **conn_bi_direct_read** (*int*) – connections that read at
        least 10x more than wrote

      * **conn_bi_direct_write** (*int*) – connections that wrote at
        least 10x more than read

      * **conn_bi_direct_both** (*int*) – remaining connections

   **Bytes read/written for relayed traffic:**

   Variables:
      * **read_history_end** (*datetime*) – end of the sampling
        interval

      * **read_history_interval** (*int*) – seconds per interval

      * **read_history_values** (*list*) – bytes read during each
        interval

      * **write_history_end** (*datetime*) – end of the sampling
        interval

      * **write_history_interval** (*int*) – seconds per interval

      * **write_history_values** (*list*) – bytes written during
        each interval

   **Cell relaying statistics:**

   Variables:
      * **cell_stats_end** (*datetime*) – end of the period when
        stats were gathered

      * **cell_stats_interval** (*int*) – length in seconds of the
        interval

      * **cell_processed_cells** (*list*) – measurement of processed
        cells per circuit

      * **cell_queued_cells** (*list*) – measurement of queued cells
        per circuit

      * **cell_time_in_queue** (*list*) – mean enqueued time in
        milliseconds for cells

      * **cell_circuits_per_decile** (*int*) – mean number of
        circuits in a decile

   **Directory Mirror Attributes:**

   Variables:
      * **dir_stats_end** (*datetime*) – end of the period when
        stats were gathered

      * **dir_stats_interval** (*int*) – length in seconds of the
        interval

      * **dir_v2_ips** (*dict*) – mapping of locales to rounded
        count of requester ips

      * **dir_v3_ips** (*dict*) – mapping of locales to rounded
        count of requester ips

      * **dir_v2_share** (*float*) – percent of total directory
        traffic it expects to serve

      * **dir_v3_share** (*float*) – percent of total directory
        traffic it expects to serve

      * **dir_v2_requests** (*dict*) – mapping of locales to rounded
        count of requests

      * **dir_v3_requests** (*dict*) – mapping of locales to rounded
        count of requests

      * **dir_v2_responses** (*dict*) – mapping of "DirResponse" to
        their rounded count

      * **dir_v3_responses** (*dict*) – mapping of "DirResponse" to
        their rounded count

      * **dir_v2_responses_unknown** (*dict*) – mapping of
        unrecognized statuses to their count

      * **dir_v3_responses_unknown** (*dict*) – mapping of
        unrecognized statuses to their count

      * **dir_v2_direct_dl** (*dict*) – mapping of "DirStat" to
        measurement over DirPort

      * **dir_v3_direct_dl** (*dict*) – mapping of "DirStat" to
        measurement over DirPort

      * **dir_v2_direct_dl_unknown** (*dict*) – mapping of
        unrecognized stats to their measurement

      * **dir_v3_direct_dl_unknown** (*dict*) – mapping of
        unrecognized stats to their measurement

      * **dir_v2_tunneled_dl** (*dict*) – mapping of "DirStat" to
        measurement over ORPort

      * **dir_v3_tunneled_dl** (*dict*) – mapping of "DirStat" to
        measurement over ORPort

      * **dir_v2_tunneled_dl_unknown** (*dict*) – mapping of
        unrecognized stats to their measurement

      * **dir_v3_tunneled_dl_unknown** (*dict*) – mapping of
        unrecognized stats to their measurement

   **Bytes read/written for directory mirroring:**

   Variables:
      * **dir_read_history_end** (*datetime*) – end of the sampling
        interval

      * **dir_read_history_interval** (*int*) – seconds per interval

      * **dir_read_history_values** (*list*) – bytes read during
        each interval

      * **dir_write_history_end** (*datetime*) – end of the sampling
        interval

      * **dir_write_history_interval** (*int*) – seconds per
        interval

      * **dir_write_history_values** (*list*) – bytes read during
        each interval

   **Guard Attributes:**

   Variables:
      * **entry_stats_end** (*datetime*) – end of the period when
        stats were gathered

      * **entry_stats_interval** (*int*) – length in seconds of the
        interval

      * **entry_ips** (*dict*) – mapping of locales to rounded count
        of unique user ips

   **Exit Attributes:**

   Variables:
      * **exit_stats_end** (*datetime*) – end of the period when
        stats were gathered

      * **exit_stats_interval** (*int*) – length in seconds of the
        interval

      * **exit_kibibytes_written** (*dict*) – traffic per port (keys
        are ints or ‘other’)

      * **exit_kibibytes_read** (*dict*) – traffic per port (keys
        are ints or ‘other’)

      * **exit_streams_opened** (*dict*) – streams per port (keys
        are ints or ‘other’)

   **Hidden Service Attributes:**

   Variables:
      * **hs_stats_end** (*datetime*) – end of the sampling interval

      * **hs_rend_cells** (*int*) – rounded count of the RENDEZVOUS1
        cells seen

      * **hs_rend_cells_attr** (*int*) – ***** attributes provided
        for the hs_rend_cells

      * **hs_dir_onions_seen** (*int*) – rounded count of the
        identities seen

      * **hs_dir_onions_seen_attr** (*int*) – ***** attributes
        provided for the hs_dir_onions_seen

   **Padding Count Attributes:**

   Variables:
      * **padding_counts** (*dict*) – ***** padding parameters

      * **padding_counts_end** (*datetime*) – end of the period when
        padding data is being collected

      * **padding_counts_interval** (*int*) – length in seconds of
        the interval

   **Bridge Attributes:**

   Variables:
      * **bridge_stats_end** (*datetime*) – end of the period when
        stats were gathered

      * **bridge_stats_interval** (*int*) – length in seconds of the
        interval

      * **bridge_ips** (*dict*) – mapping of locales to rounded
        count of unique user ips

      * **geoip_start_time** (*datetime*) – replaced by
        bridge_stats_end (deprecated)

      * **geoip_client_origins** (*dict*) – replaced by bridge_ips
        (deprecated)

      * **ip_versions** (*dict*) – mapping of ip protocols to a
        rounded count for the number of users

      * **ip_versions** – mapping of ip transports to a count for
        the number of users

   ***** attribute is either required when we’re parsed with
   validation or has a default value, others are left as **None** if
   undefined

   Changed in version 1.4.0: Added the hs_stats_end, hs_rend_cells,
   hs_rend_cells_attr, hs_dir_onions_seen, and hs_dir_onions_seen_attr
   attributes.

   Changed in version 1.6.0: Added the padding_counts,
   padding_counts_end, and padding_counts_interval attributes.

   digest(hash_type='SHA1', encoding='HEX')

      Digest of this descriptor’s content. These are referenced by…

         * **Server Descriptors**

           * Referer: "ServerDescriptor" **extra_info_digest**
             attribute

           * Format: **SHA1/HEX**

         * **Server Descriptors**

           * Referer: "ServerDescriptor"
             **extra_info_sha256_digest** attribute

           * Format: **SHA256/BASE64**

      Changed in version 1.8.0: Added the hash_type and encoding
      arguments.

      Parameters:
         * **hash_type** (*stem.descriptor.DigestHash*) – digest
           hashing algorithm

         * **encoding** (*stem.descriptor.DigestEncoding*) – digest
           encoding

      Returns:
         **hashlib.HASH** or **str** based on our encoding argument

class stem.descriptor.extrainfo_descriptor.RelayExtraInfoDescriptor(raw_contents, validate=False)

   Bases: "stem.descriptor.extrainfo_descriptor.ExtraInfoDescriptor"

   Relay extra-info descriptor, constructed from data such as that
   provided by ‘GETINFO extra-info/digest/*’, cached descriptors, and
   metrics (specification).

   Variables:
      * **str** (*ed25519_signature*) – base64 encoded ed25519
        certificate

      * **str** – signature of this document using ed25519

      * **signature** (*str*) – ***** signature for this extrainfo
        descriptor

   ***** attribute is required when we’re parsed with validation

   Changed in version 1.5.0: Added the ed25519_certificate and
   ed25519_signature attributes.

   TYPE_ANNOTATION_NAME = 'extra-info'

   classmethod content(attr=None, exclude=(), sign=False, signing_key=None)

      Creates descriptor content with the given attributes. Mandatory
      fields are filled with dummy information unless data is
      supplied. This doesn’t yet create a valid signature.

      New in version 1.6.0.

      Parameters:
         * **attr** (*dict*) – keyword/value mappings to be included
           in the descriptor

         * **exclude** (*list*) – mandatory keywords to exclude from
           the descriptor, this results in an invalid descriptor

         * **sign** (*bool*) – includes cryptographic signatures and
           digests if True

      Returns:
         **str** with the content of a descriptor

      Raises:
         * **ImportError** if cryptography is unavailable and sign
           is True

         * **NotImplementedError** if not implemented for this
           descriptor type

   classmethod create(attr=None, exclude=(), validate=True, sign=False, signing_key=None)

      Creates a descriptor with the given attributes. Mandatory fields
      are filled with dummy information unless data is supplied. This
      doesn’t yet create a valid signature.

      New in version 1.6.0.

      Parameters:
         * **attr** (*dict*) – keyword/value mappings to be included
           in the descriptor

         * **exclude** (*list*) – mandatory keywords to exclude from
           the descriptor, this results in an invalid descriptor

         * **validate** (*bool*) – checks the validity of the
           descriptor’s content if **True**, skips these checks
           otherwise

         * **sign** (*bool*) – includes cryptographic signatures and
           digests if True

      Returns:
         "Descriptor" subclass

      Raises:
         * **ValueError** if the contents is malformed and validate
           is True

         * **ImportError** if cryptography is unavailable and sign
           is True

         * **NotImplementedError** if not implemented for this
           descriptor type

   digest

class stem.descriptor.extrainfo_descriptor.BridgeExtraInfoDescriptor(raw_contents, validate=False)

   Bases: "stem.descriptor.extrainfo_descriptor.ExtraInfoDescriptor"

   Bridge extra-info descriptor (bridge descriptor specification)

   Variables:
      * **ed25519_certificate_hash** (*str*) – sha256 hash of the
        original identity-ed25519

      * **router_digest_sha256** (*str*) – sha256 digest of this
        document

   Changed in version 1.5.0: Added the ed25519_certificate_hash and
   router_digest_sha256 attributes.

   TYPE_ANNOTATION_NAME = 'bridge-extra-info'

   classmethod content(attr=None, exclude=(), sign=False)

      Creates descriptor content with the given attributes. Mandatory
      fields are filled with dummy information unless data is
      supplied. This doesn’t yet create a valid signature.

      New in version 1.6.0.

      Parameters:
         * **attr** (*dict*) – keyword/value mappings to be included
           in the descriptor

         * **exclude** (*list*) – mandatory keywords to exclude from
           the descriptor, this results in an invalid descriptor

         * **sign** (*bool*) – includes cryptographic signatures and
           digests if True

      Returns:
         **str** with the content of a descriptor

      Raises:
         * **ImportError** if cryptography is unavailable and sign
           is True

         * **NotImplementedError** if not implemented for this
           descriptor type

   digest(hash_type='SHA1', encoding='HEX')

      Digest of this descriptor’s content. These are referenced by…

         * **Server Descriptors**

           * Referer: "ServerDescriptor" **extra_info_digest**
             attribute

           * Format: **SHA1/HEX**

         * **Server Descriptors**

           * Referer: "ServerDescriptor"
             **extra_info_sha256_digest** attribute

           * Format: **SHA256/BASE64**

      Changed in version 1.8.0: Added the hash_type and encoding
      arguments.

      Parameters:
         * **hash_type** (*stem.descriptor.DigestHash*) – digest
           hashing algorithm

         * **encoding** (*stem.descriptor.DigestEncoding*) – digest
           encoding

      Returns:
         **hashlib.HASH** or **str** based on our encoding argument
