Mirrors/naiveproxy
1
0
Fork 0
mirror of https://github.com/klzgrad/naiveproxy.git synced 2026-06-01 20:06:31 +03:00
Code Issues Packages Projects Releases Wiki Activity

Import chromium-123.0.6312.40

Browse Source
This commit is contained in:
importer
2024-03-21 18:13:28 +08:00
committed by klzgrad
commit f0de60c19c
24941 changed files with 4484504 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

184
src/net/docs/adding_doh_providers.md Normal file
View File

@@ -0,0 +1,184 @@
# Adding/Modifying DoH providers
Chrome performs autoupgrade of Classic DNS resolvers to equivalent same-provider
DoH servers using the mapping encoded in
[`DohProviderEntry::GetList()`](/net/dns/public/doh_provider_entry.cc).
[TOC]
## Requesting provider list changes
Official representatives of a DoH provider can request the addition of or
modifications to the entry for their service. See the
[separate documentation](https://docs.google.com/document/d/128i2YTV2C7T6Gr3I-81zlQ-_Lprnsp24qzy_20Z1Psw)
for the criteria and process.
After following the process, if approved, the addition or modification will be
made by a member of the Chromium team.
## Modifying the DoH provider list
*** note
All modifications to the DoH provider list must be accompanied by a request in
the Chromium bug tracker in the
[DoH component](https://bugs.chromium.org/p/chromium/issues/list?q=component:Internals%3ENetwork%3EDoH),
and that request must be approved by the Chrome team.
It is generally expected that the actual code change will be performed by the
Chrome team on approval of the request.
***
1. Ensure the request bug has been approved with a comment from a member of the
Chrome team indicating the approval in the bug. Confirm whether the approved
request is for an autoupgrade mapping, inclusion in the Chrome "Secure DNS"
settings UI, or both.
It is not necessary for the author or reviewer of the CL modifying the
provider list to verify any aspects of the provider criteria, including
ownership of hostnames. That is the responsibility of the Chrome team member
who approved the request bug.
1. Add or modify the [`DohProviderEntry`](/net/dns/public/doh_provider_entry.h)
entry in
[`DohProviderEntry::GetList()`](/net/dns/public/doh_provider_entry.cc).
* `provider` is a unique string label used to identify the specific entry.
It may be used for provider-specific metrics data sent to Google. The
`provider` string should be camelcase, and for cases where a single
organization runs multiple servers/varieties, the overall organization
name should go before the variety-specific name. For example, if
ExampleOrg has both filtered and unfiltered servers, they may have two
provider list entries with the `provider` strings "ExampleOrgFiltered"
and "ExampleOrgUnfiltered".
* `feature` is used by experiments to control an individual provider. When
the feature is disabled, either by default or remotely by an experiment
config, the provider is not eligible for autoupgrade and it will not be
listed in the "Secure DNS" settings.
* `ip_addresses` is the list of Classic DNS server IP addresses that are
eligible for upgrade to the provider\'s DoH server. The addresses do not
need to be unique within the overall provider list. If multiple DoH
provider entries contain the same `ip_addresses` entry, the DoH servers
for all containing provider entries could become available for use if
Chrome detects that IP address configured. `ip_addresses` may also be
empty if a provider is only available in "Secure DNS" settings and not
for autoupgrade.
* `dns_over_tls_hostnames` is used for autoupgrade from DoT to DoH. May
be used on platforms where Chrome can recognize configured DoT servers.
Similar to `ip_addresses` in that it may be empty or non-unique. Note
that addition/modification requests in the bug tracker often forget to
mention DoT hostnames, so be sure to ask about it if you suspect a DoH
provider may have an equivalent DoT endpoint.
* `dns_over_https_template` is the URI template of the DoH server. It is
formatted according to [RFC6570](https://tools.ietf.org/html/rfc6570)
and [RFC8484](https://tools.ietf.org/html/rfc8484). If the template
contains a single `dns` variable, Chrome will perform GET requests, and
if the template contains no variables, Chrome will perform POST
requests. Confirm this matches with the provider's GET/POST preference
in the bug tracker request.
* `ui_name` is the name that will be displayed to users in the "Secure
DNS" settings. It is only needed for providers that will be listed in
the settings. Confirm that the name conforms to the rules in the
[criteria document](https://docs.google.com/document/d/128i2YTV2C7T6Gr3I-81zlQ-_Lprnsp24qzy_20Z1Psw/edit#heading=h.l3wtx3cufz78).
***promo
Note that all `ui_name` values are currently ASCII-only strings. While
non-ASCII names make sense, especially for region-specific providers,
and there is no known issue with using such names, support has not been
thoroughly tested. If adding a non-ASCII name, take extra care to test
that it displays correctly in settings UIs for all platforms.
***
* `privacy_policy` is a URL to the privacy policy page for the provider.
It is only needed for providers that will be listed in the settings.
* `display_globally` sets whether or not a provider will appear in the
"Secure DNS" settings globally for all users. This flag is only expected
to be set for a small number of providers.
* `display_countries` sets any specific countries in which the provider
should appear in "Secure DNS" settings. Should be empty if
`display_globally` is set. Format is the two-letter ISO 3166-1 country
codes. The provider will be displayed to users when the OS region
settings consider the OS to be in one of the given countries.
* `logging_level` should be set to `kExtra` for any entry for which
logging/monitoring/etc is especially important. This should be the case
only for the couple most-used providers in the list, newly-entered
providers with some risk of issues, or providers with a history of
issues requiring that provider to be disabled for auto upgrade.
1. Manually test the new addition/modification.
*** promo
If running tests on enterprise-maintained machines, DoH may be disabled,
leading to DoH tests always failing and the "Secure DNS" settings being
disabled. In such cases, a strategic local-only modification to
[`StubResolverConfigReader`](/chrome/browser/net/stub_resolver_config_reader.cc)
may be necessary to bypass the disabling.
***
* After making any additions or modifications to the provider list, run
the DoH browser tests:
```shell
browser_tests.exe --run-manual \
--gtest_filter= DohBrowserParameterizedTest/DohBrowserTest.*
```
Investigate any failures, especially concerning the modified
provider(s).
For new providers, repeat the test 25-100 times (exercise judgment on
how much scrutiny is necessary) for the specific provider to ensure the
provider is reliable:
```shell
browser_tests.exe --run-manual \
--gtest_filter= DohBrowserParameterizedTest/DohBrowserTest.MANUAL_ExternalDohServers/PROVIDER_ID_HERE \
--gtest_repeat=100
```
* If adding/modifying a provider intended for display in "Secure DNS"
settings, load up the settings page and select/deselect the provider
followed by making some simple test requests to ensure it functions
correctly.
If the provider is only intended to be displayed in specific countries,
test the settings inside and outside those countries by modifying the OS
region settings and ensuring the entry only displays for the correct
regions. On Windows 10, this setting is found under
"Time & Language" > "Region" > "Country or region"
*** aside
TODO: Document region settings for other operating systems.
***
1. Send the list modification code review to a reviewer in
[DNS OWNERS](/net/dns/OWNERS), or if no DNS owners are available, to a
reviewer in [net OWNERS](/net/OWNERS). The reviewer should confirm that the
process defined in this document has been followed, especially that the bug
tracker request has been properly approved.
1. If a provider must be initially disabled or partially disabled, e.g. because
a provider with significant usage has requested a gradual controlled
rollout, a Googler must:
* Create a launch bug, e.g. the [Cox DoH provider launch
bug](https://crbug.com/1303146).
* Create a Finch config to roll out each DoH provider, e.g.
`DnsOverHttpsCox.gcl`.
* Ensure that the provider's `DohProviderEntry::feature` is disabled by
default and is enabled by the Finch config.
* Before landing the Finch config, make the corresponding changes in
[fieldtrial_testing_config.json](/testing/variations/fieldtrial_testing_config.json).
* Once the DoH provider's feature has been launched and the Finch
experiment has expired, `DohProviderEntry::feature` should be enabled
by default.
## Dynamic control
DoH providers, especially new ones, may have service interruptions or
performance degradation to the point that it's necessary to disable their
autoupgrade feature.
If the malfunctioning DoH provider is still in the middle of a gradual rollout,
Googlers may dynamically disable the provider by modifying its experiment config
(`DnsOverHttps${ProviderName}.gcl`).
Otherwise, if the provider's autoupgrade feature has already been launched,
Googlers should create a new "kill switch config" rather than reuse the expired
gradual rollout config. Follow the guidance at
[go/finch-killswitch](http://go/finch-killswitch).
*** aside
If a user has selected a provider via the "Secure DNS" settings and that
provider becomes disabled, the UI option will disappear from the dropdown but
selection will convert to a custom text-box entry for the same provider and
continue to be used for that user.
***

163
src/net/docs/bug-triage-labels.md Normal file
View File

@@ -0,0 +1,163 @@
# Chrome Network Bug Triage : Components and labels
## Some network component caveats
* **Internals>Network>SSL**
This includes issues that should be also tagged as **Security>UX**
(certificate error pages or other security interstitials, omnibox indicators
that a page is secure), and more general SSL issues. If you see requests
that die in the SSL negotiation phase, in particular, this is often the
correct component.
* **Internals>Network>Cache**
The cache is the layer that handles most range request logic (Though range
requests may also be issued by the PDF plugin, XHRs, or other components).
* **Internals>Network>HTTP**
Typically not used. Unclear what it covers, and there's no specific HTTP
owner.
* **Internals>Network>Logging**
Covers **about:net-internals**, **about:net-export** as well as the what's
sent to the NetLog.
* **Internals>Network>Connectivity**
Issues related to switching between networks, `ERR_NETWORK_CHANGED`, Chrome
thinking it's online when it's not / navigator.onLine inaccuracies, etc.
* **Internals>Network>Filters**
Covers gzip, deflate and brotli issues. `ERR_CONTENT_DECODING_FAILED`
indicates a problem at this layer, and bugs here can also cause response
body corruption.
## Common non-network components
Bugs in these areas often receive the **Internals>Network** component, though
they fall largely outside the purview of the network stack team:
* **UI>Browser>Downloads**
Despite the name, this covers all issues related to downloading a file
except saving entire pages (which is **Blink>SavePage**), not just UI
issues. Most downloads bugs will have the word "download" or "save as" in
the description. Issues with the HTTP server for the Chrome binaries are
not downloads bugs.
* **UI>Browser>SafeBrowsing**
Bugs that have to do with the process by which a URL or file is determined
to be dangerous based on our databases, or the resulting interstitials.
Determination of danger based purely on content-type or file extension
belongs in **UI>Browser>Downloads**, not SafeBrowsing.
* **Blink>Forms**
Issues submitting forms, forms having weird data, forms sending the wrong
method, etc.
* **Blink>Loader**
Cross origin issues are sometimes loader related. Blink also has an
in-memory cache, and when it's used, requests don't appear in
about:net-internals. Requests for the same URL are also often merged there
as well. This does *not* cover issues with content/browser/loader/ files.
* **Blink>ServiceWorker**
* **Blink>Network>WebSockets**
Issues with the WebSockets. Attach this component to any issue about the
WebSocket feature regardless of where the cause of the issue is (net/ or
Blink).
* **Blink>Network>FetchAPI**
Generic issues with the Fetch API - missing request or response headers,
multiple headers, etc. These will often run into issues in certain corner
cases (Cross origin / CORS, proxy, whatever). Attach all components that
seem appropriate.
* **Blink>Network>XHR**
Generic issues with sync/async XHR requests.
* **Blink>WebRTC>Network**
Anything WebRTC-related does not use the net stack and should go here.
* **Services>Sync**
Sharing data/tabs/history/passwords/etc between machines not working.
* **Services>Chromoting**
* **Platform>Extensions**
Issues extensions loading / not loading / hanging.
* **Platform>Extensions>API**
Issues with network related extension APIs should have this component.
chrome.webRequest is the big one, I believe, but there are others.
* **Internals>Plugins>Pepper[>SDK]**
* **UI>Browser>Omnibox**
Basically any issue with the omnibox. URLs being treated as search queries
rather than navigations, dropdown results being weird, not handling certain
Unicode characters, etc. If the issue is new TLDs not being recognized by
the omnibox, that's due to Chrome's TLD list being out of date, and not an
omnibox issue. Such TLD issues should be duped against
http://crbug.com/37436.
* **Internals>Media>Network**
Issues related to media. These often run into the 6 requests per hostname
issue, and also have fun interactions with the cache, particularly in the
range request case.
* **Internals>Plugins>PDF**
Issues loading PDF files. These are often related to range requests, which
also have some logic at the Internals>Network>Cache layer.
* **UI>Browser>Navigation**
Despite the name, this covers all issues related to page navigation, not
just UI issues.
* **UI>Browser>History**
Issues which only appear with forward/back navigation.
* **OS>Systems>Network** / **OS>Systems>Mobile** / **OS>Systems>Bluetooth**
These should be used for issues with Chrome OS's platform network code, and
not net/ issues on Chrome OS.
* **Blink>SecurityFeature**
CORS / Cross origin issues. Main frame cross-origin navigation issues are
often actually **UI>Browser>Navigation** issues.
* **Privacy**
Privacy related bug (History, cookies discoverable by an entity that
shouldn't be able to do so, incognito state being saved in memory or on disk
beyond the lifetime of incognito tabs, etc). Generally used in conjunction
with other components.
## Common labels
* **Type-Bug-Security**
Security related bug (Allows for code execution from remote site, allows
crossing security boundaries, unchecked array bounds, etc) should be tagged
with this label.

168
src/net/docs/bug-triage.md Normal file
View File

@@ -0,0 +1,168 @@
# Chrome Network Bug Triage
The Chrome network team uses a two day bug triage rotation. The goal is to
review outstanding issues and keep things moving forward. The rotation is time
based rather than objective based. Sheriffs are expected to spend the majority
of their two days working on bug triage/investigation.
## 1. Review untriaged bugs
Look through [this list of untriaged
bugs](https://bugs.chromium.org/p/chromium/issues/list?sort=pri%20-stars%20-opened&q=status%3Aunconfirmed%2Cuntriaged%20-Needs%3DFeedback%20-Label%3ANetwork-Triaged%20-has%3ANextAction%20component%3DInternals%3ENetwork%3EReportingAndNEL%2CInternals%3ENetwork%3ECache%3ESimple%2CInternals%3ENetwork%2CInternals%3ENetwork%3ECache%2CInternals%3ENetwork%3ESSL%2CInternals%3ENetwork%3EQUIC%2CInternals%3ENetwork%3EAuth%2CInternals%3ENetwork%3EHTTP2%2CInternals%3ENetwork%3EProxy%2CInternals%3ENetwork%3ELogging%2CInternals%3ENetwork%3EConnectivity%2CInternals%3ENetwork%3EDomainSecurityPolicy%2CInternals%3ENetwork%3EFTP%2CInternals%3ENetwork%3EDNS).
The goal is for this query to be empty. Bugs can be removed from the triage queue
by doing any of the following:
* Changing the bug status - marking the bug Available, or closing it.
* Removing the `Internals>Network` component or subcomponent.
* Adding the label `Network-Triaged` (when there are multiple components).
For each bug try to:
* Remove the `Internals>Network` component or subcomponent if it belongs
elsewhere
* Dupe it against an existing bug
* Close it as `WontFix`.
* Give the bug a priority. Refer to [this document for guidelines](https://docs.google.com/document/d/1JOtp1LS7suqTjMuv41jQFc7aCTR33zJKPoGjKpvVFCA)
* If the bug is a potential security issue (Allows for code execution from remote
site, allows crossing security boundaries, unchecked array bounds, etc) mark
it `Type-Bug-Security`.
* If the bug has privacy implications mark it with component `Privacy`.
* Set the type to `Task` or `Feature` when it is not a bug.
* Pay extra attention to possible regressions. Ask the reporter to narrow down using
[bisect-builds-py](https://www.chromium.org/developers/bisect-builds-py). To
view suspicious changelists in a regression window, you can use the Change Log
form on [OmahaProxy](https://omahaproxy.appspot.com/)
* CC others who may be able to help
* Mark it as `Needs-Feedback` and request more information if needed.
* In cases where the bug has multiple components, but primary ownership falls
outside of networking, further network triage may not be possible. In those
cases, if possible remove the networking component. Otherwise, add the
`Network-Triaged` label to the bug, and add a comment explaining which team
should triage further. Adding the `Network-Triaged` serves to filter the
bug from our untriaged bug list.
* Avoid spending time deep-diving into ambiguous issues when you suspect it is
an out of scope server or network problem, and is not clearly high priority
(for instance, it affects only 1 user and is not a regression).
Instead:
* Mark the bug as `Available` with Priority 3.
* Add the `Needs-Feedback` label
* Add a comment thanking the reporter, but explaining the issue is ambiguous
and they need to do the debugging to demonstrate it is an actual Chrome bug.
* Point them to `chrome://net-export` and the
[NetLog Viewer](https://netlog-viewer.appspot.com/).
* Ask them to confirm whether it is a Chromium regression. (Regressions are
treated as high priority)
* Request a NetLog that captures the problem. You can paste this on the bug:
```
Please collect and attach a chrome://net-export log.
Instructions can be found here:
https://chromium.org/for-testers/providing-network-details
```
* If a NetLog was provided, try to spend a bit of time reviewing it. See
[crash-course-in-net-internals.md](crash-course-in-net-internals.md) for an
introduction.
* Move to a subcomponent of `Internals>Network` if appropriate. See
[bug-triage-labels.md](bug-triage-labels.md) for an overview of the components.
* If the bug is a crash, see [internal: Dealing with a crash
ID](https://goto.google.com/network_triage_internal#dealing-with-a-crash-id)
and [internal: Investigating
crashers](https://goto.google.com/network_triage_internal#investigating-crashers)
## 2. Follow-up on issues with the Needs-Feedback label
Look through [this list of Needs=Feedback
bugs](https://bugs.chromium.org/p/chromium/issues/list?sort=-modified%20-modified&q=Needs%3DFeedback%20component%3DInternals%3ENetwork%3EReportingAndNEL%2CInternals%3ENetwork%3ECache%3ESimple%2CInternals%3ENetwork%2CInternals%3ENetwork%3ECache%2CInternals%3ENetwork%3ESSL%2CInternals%3ENetwork%3EQUIC%2CInternals%3ENetwork%3EAuth%2CInternals%3ENetwork%3EHTTP2%2CInternals%3ENetwork%3EProxy%2CInternals%3ENetwork%3ELogging%2CInternals%3ENetwork%3EConnectivity%2CInternals%3ENetwork%3EDomainSecurityPolicy%2CInternals%3ENetwork%3EFTP%2CInternals%3ENetwork%3EDNS).
* If the requested feedback was provided, review the new information and repeat
the same steps as (1) to re-triage based on the new information.
* If the bug had the `Needs-Feedback` label for over 30 days, and the
feedback needed to make progress was not yet provided, archive the bug.
## 3. Ensure P0 and P1 bugs have an owner
Look through [the list of unowned high priority
bugs](https://bugs.chromium.org/p/chromium/issues/list?sort=pri%20-stars%20-opened&q=Pri%3A0%2C1%20-has%3Aowner%20-label%3ANetwork-Triaged%20component%3DInternals%3ENetwork%3EReportingAndNEL%2CInternals%3ENetwork%3ECache%3ESimple%2CInternals%3ENetwork%2CInternals%3ENetwork%3ECache%2CInternals%3ENetwork%3ESSL%2CInternals%3ENetwork%3EQUIC%2CInternals%3ENetwork%3EAuth%2CInternals%3ENetwork%3EHTTP2%2CInternals%3ENetwork%3EProxy%2CInternals%3ENetwork%3ELogging%2CInternals%3ENetwork%3EConnectivity%2CInternals%3ENetwork%3EDomainSecurityPolicy%2CInternals%3ENetwork%3EFTP%2CInternals%3ENetwork%3EDNS).
These bugs should either have an owner, or be downgraded to a lower priority.
## 4. (Optional) Look through crash reports
Top crashes will already be entered into the bug system by a different process,
so will be handled by the triage steps above.
However if you have time to look through lower threshold crashes, see
[internal: Looking for new crashers](https://goto.google.com/network_triage_internal#looking-for-new-crashers)
## 5. Send out a sheriff report
On the final day of your rotation, send a brief summary to net-dev@chromium.org
detailing any interesting or concerning trends. Do not discuss any restricted
bugs on the public mailing list.
## Covered bug components
Not all of the subcomponents of `Interals>Network` are handled by this rotation.
The ones that are included are:
```
Internals>Network
Internals>Network>Auth
Internals>Network>Cache
Internals>Network>Cache>Simple
Internals>Network>DNS
Internals>Network>Connectivity
Internals>Network>DomainSecurityPolicy
Internals>Network>FTP
Internals>Network>HTTP2
Internals>Network>Logging
Internals>Network>Proxy
Internals>Network>QUIC
Internals>Network>ReportingAndNEL
Internals>Network>SSL
```
The rest of the `Internals>Network` subcomponents are out of scope,
and covered by separate rotations:
```
Internals>Network>Certificate
Internals>Network>CertTrans
Internals>Network>Cookies
Internals>Network>DataProxy
Internals>Network>DataUse
Internals>Network>DoH
Internals>Network>EV
Internals>Network>Library
Internals>Network>NetInfo
Internals>Network>NetworkQuality
Internals>Network>TrustTokens
Internals>Network>VPN
```
## Management
* Your rotation will appear in Google Calendar for three days, as a full day
event. However you should work on it during normal business hours only.
The bug triage work should be considered P1 during this period, but you
should feel free to work on project work once you triage outstanding issues.
* Google Calendar [c_c76b86b0356db19c1e06879b16d84a2fb7b9747f066b671c46875261c9e7f17a@group.calendar.google.com](https://calendar.google.com/calendar/embed?src=c_c76b86b0356db19c1e06879b16d84a2fb7b9747f066b671c46875261c9e7f17a%40group.calendar.google.com&ctz=Asia%2FTokyo)
* Owners for the network bug triage rotation can find instructions on
generating and modifying shifts
[here (internal-only)](https://goto.google.com/net-triage-setup).
* An overview of bug trends can be seen on [Chromium
Dashboard](https://chromiumdash.appspot.com/components/Internals/Network?project=Chromium)
* The issue tracker doesn't track any official mappings between components and
OWNERS. This [internal document](https://goto.google.com/kojfj) enumerates
the known owners for subcomponents. Owners information is dynamic, and the
document might become outdated, you can always go to the source, search for
the component in a DIR_METADATA file and look for an OWNERS file in the same,
or parent directory.
* [Web Platform Team SLOs](https://docs.google.com/document/d/18ylPve6jd43m8B7Dil6xmS4G9MHL2_DhQon72je-O9o/edit)
* [Web Platform bug triage guidelines](https://docs.google.com/document/d/1JOtp1LS7suqTjMuv41jQFc7aCTR33zJKPoGjKpvVFCA)

385
src/net/docs/certificate-transparency.md Normal file
View File

@@ -0,0 +1,385 @@
# Certificate Transparency
[TOC]
## Overview
Certificate Transparency (CT) is a protocol designed to fix several structural
flaws in the SSL/TLS certificate ecosystem. Described in
[RFC 6962](https://tools.ietf.org/html/rfc6962), it provides a public,
append-only data structure that can log certificates that are issued by
[certificate authorities](https://en.wikipedia.org/wiki/Certificate_authority) (CAs).
By logging certificates, it becomes possible for the public to see what
certificates have been issued by a given CA. This allows site operators to
detect when a certificate has been issued for their domains, allowing them to
check for unauthorized issuance. It also allows browsers and root stores, and
the broader community, to examine the certificates a CA has issued and ensure
that the CA is complying with their expected or disclosed practices.
For more information about how Certificate Transparency works, see:
* https://www.certificate-transparency.org
* [Introducing Certificate Transparency and Nimbus](https://blog.cloudflare.com/introducing-certificate-transparency-and-nimbus/)
## Certificate Transparency for Site Operators
### Basics
We say that a certificate supports Certificate Transparency if it comes with
CT information that demonstrates it has been logged in several CT logs. This
CT information must comply with the
[Certificate Transparency in Chrome](https://github.com/chromium/ct-policy/blob/master/ct_policy.md)
policy. We sometimes refer to a site that "supports" CT as using a certificate
that is "CT qualified" or "disclosed via CT."
In general, a site operator does not need to take special action to
support Certificate Transparency. This is because RFC 6962 defines three ways
of providing the necessary information for CT: within the certificate, within
a stapled OCSP response, or directly by the TLS server. Nearly every CA
supports CT through the first method, meaning that when you get a certificate,
it will already support CT and require no further configuration. If you are
using a cloud provider to terminate your TLS connections, the cloud provider
may also support CT via TLS, requiring no further action on your part.
Supporting CT within the certificate itself is the preferred and recommended
way to enable CT support. If you obtain a certificate from your CA and it does
not support CT, then that generally indicates that your CA is not following
industry best practice, and you should probably look for another CA to provide
certificates for your sites.
Configuring support for CT via the TLS extension is not recommended for most
site operators. This is because supporting CT via this method requires
constant monitoring of the CT ecosystem, such as for changes in the list of
trusted logs or testing compatibility with various CT-supporting clients. This
method works well for organizations with the ability to dedicate resources to
that, such as hosting and cloud providers. If you are hosting your own website,
you should try to ensure that your certificates support CT, and avoid supporting
CT via the TLS extension. Supporting CT via the TLS extension may require rapid
changes to your configuration, and thus may be riskier for organizations
without staff dedicated to this.
If you are getting longer-lived certificates (for example, 1 year), it's
possible that changes in the CT ecosystem may mean that the CT information may
expire before the certificate expires. If your CA also supports delivering CT
via OCSP responses, then supporting OCSP stapling on your server may allow
fresh CT information to be provided without having to replace the certificate.
Alternatively, if your server does not support OCSP stapling, or your CA does
not support CT in their OCSP responses, you may need to replace your certificate.
These policies only apply to publicly-trusted CAs - that is, CAs that your
browser or device trust without any additional configuration. For organizations
using their own CAs, or for locally installed CAs, see
[Certificate Transparency for Enterprises](#Certificate-Transparency-For-Enterprises).
### Chrome Policies
Chrome has gradually required Certificate Transparency for more and more
publicly-trusted certificates over the past few years.
* [Since 1 January 2015](https://github.com/chromium/ct-policy/blob/master/ct_policy.md),
Chrome has required that all Extended Validation certificates be disclosed via
Certificate Transparency. Certificates that were not properly disclosed would
be [stripped of their EV status](https://news.netcraft.com/archives/2015/08/24/thousands-short-changed-by-ev-certificates-that-dont-display-correctly-in-chrome.html),
but no warnings would be shown to visitors to sites that did not comply.
* [Since 1 June 2016](https://security.googleblog.com/2015/10/sustaining-digital-certificate-security.html),
Chrome has required that all new certificates issued by the set of root
certificates owned by Symantec Corporation are disclosed via Certificate
Transparency. Certificates that were not disclosed, or which were not disclosed
in a way consistent with RFC 6962, would be rejected as untrusted.
* For all new certificates issued after 30 April 2018, [Chrome will require that
the certificate be disclosed via Certificate
Transparency](https://groups.google.com/a/chromium.org/d/msg/ct-policy/wHILiYf31DE/iMFmpMEkAQAJ).
If a certificate is issued after this date and neither the certificate nor
the site supports CT, then these certificates will be rejected as untrusted, and
the connection will be blocked. In the case of a main page load, the user will
see a full page certificate warning page, with the error code
`net::ERR_CERTIFICATE_TRANSPARENCY_REQUIRED`. If you receive this error, this
indicates that your CA has not taken steps to make sure your certificate
supports CT, and you should contact your CA's sales or support team to ensure
you can get a replacement certificate that works.
### Domain Privacy
Supporting CT by disclosing the certificate to a CT Log means that the full
contents of the certificate will be publicly accessible and viewable. In
particular, this means that the domains a certificate are for will be included
in the Certificate Transparency log, as well as the organization they are
affiliated with, if they are validated to a level higher than Domain
Validation or issued from an organization-specific CA.
For most certificates, this is no different than what is already available.
Publicly-trusted certificates have been subject to aggregation for public
analysis for some time, such as through products and tools such as
[Censys](https://censys.io/) or [scans.io](https://scans.io/). While
Certificate Transparency provides an interoperable protocol for exchanging
these datasets, in many cases, the certificate details and domains were already
publicly detectable.
Requiring that the full certificate be disclosed if it was issued by a
publicly-trusted CA is an important part of the security goals of Certificate
Transparency. Permitting some of the information to be hidden from
certificates allows for both attackers and untrustworthy CAs to hide
certificates that could be used to compromise users. Certificate Transparency
has detected issues at a large
[number of CAs](https://wiki.mozilla.org/CA/Incident_Dashboard), many that the
CAs themselves were not even aware of, and so public disclosure is critical
to keeping all users safe.
While proposals for hiding domain names were presented during the development
of Certificate Transparency, none of them were able to balance the needs of
site operators that did not need to hide their domains, those that did, and the
security risks that users would face.
Because of this, Chrome does not support any method for hiding domain names or
other information within publicly-trusted certificates, nor are there any plans
to support such mechanisms. Domain operators that wish to hide their
certificates, enabling security risks and attacks, have two options:
1. **Wildcard Certificates** - Wildcard certificates allow a single certificate
to be used for multiple hostnames, by putting a `*` as the most specific
DNS label (for example, `*.internal.example.com` is valid for
`mail.internal.example.com` and `wiki.internal.example.com`, but not for
`www.example.com` or `two.levels.internal.example.com`). Wildcard
certificates require greater care by the site operator to protect their
private key, but also can have their issuance controlled via technologies
such as [CAA (RFC 6844)](https://tools.ietf.org/html/rfc6844). This still
requires the certificate be disclosed, but can limit how much of the domain
is disclosed.
2. **Enterprise-specific configuration** - If the domains being accessed are
not intended to be used on the public internet, or not on machines or by
users that are not part of a single enterprise, then that enterprise can
use the options in the
[Certificate Transparency for Enterprises](#Certificate-Transparency-For-Enterprises).
This allows the enterprise to not reveal any information about the
certificate, but these certificates will **only** be trusted by their
members.
### What to do if your certificate does not work
As noted in [Chrome Policies](#Chrome-Policies), all certificates issued after
30 April 2018 are expected to be disclosed via Certificate Transparency in a
way that is compliant with the Certificate Transparency in Chrome policy.
Virtually all publicly-trusted CAs have committed to supporting CT for their
customers by default by this date, meaning that site operators should not have
to do anything special and can continue getting certificates that just work on
1 May 2018.
However, there's still a chance that a CA may not have adopted Certificate
Transparency, may have an infrastructure issue, or may not have communicated
to their partners, such as resellers or subordinate CAs, to ensure that the
transition would be as smooth as possible for their customers.
If you're receiving a `net::ERR_CERTIFICATE_TRANSPARENCY_REQUIRED` error
message, the best thing to do is to contact your CA's support or sales team
to diagnose the error with them. They will most likely need to replace your
certificate with a new one that properly supports CT.
## Certificate Transparency for Enterprises
### Locally-trusted CAs
Certificate Transparency only applies to CAs that are publicly-trusted - that
is, CAs that are supported by your browser or device out of the box, without
any additional configuration steps.
For CAs that have been manually installed, provided those certificates are not
or have not been publicly-trusted, it's not necessary to enable support for
Certificate Transparency. Further, Certificate Transparency Logs will not
accept certificates from those CAs, thus it's not possible to support CT.
In some cases, an Enterprise may have a locally-trusted CA that has been
manually installed, but it was previously publicly-trusted. For example, this
CA may have been removed by a browser or an OS for not complying with the
root store policies, but the Enterprise may still have a dependency on
trusting this CA. In these cases, the Enterprise can use
[Enterprise Policies](#Enterprise-Policies) to configure how Certificate
Transparency will be enforced for those CAs.
### Private Domain Names
For Enterprises that have domain names that are internal to their organization,
and do not need to be publicly-trusted by default, several options exist to
enable these domains to be kept private, while allowing the certificates to
still be used, without error, for users in their organization.
The recommended option is to no longer rely on publicly-trusted certificates
to serve these domains, as they are organization specific. For example, such
organizations can use a private CA, which [several](https://aws.amazon.com/certificate-manager/private-certificate-authority/)
[CAs](https://www.digicert.com/private-pki/) [offer](https://www.comodo.com/business-security/pki-management/certificate-manager.php).
Using a hosted, managed PKI may help organizations more rapidly respond to
change in the TLS ecosystem, such as changes to certificate algorithms or
support for new protocols.
Another option is to request that the publicly-trusted CA not log the
certificate. This will prevent this certificate from being trusted by default,
but organizations that manage their devices or users can override this through
[Enterprise Policies](#Enterprise-Policies) to enable these certificates to be
trusted for users in their Enterprise.
Finally, organizations may manage their own PKI in-house, using CA
software such as [CFSSL](https://github.com/cloudflare/cfssl), [Boulder](https://github.com/letsencrypt/boulder),
[EJBCA](https://www.ejbca.org/) or
[Active Directory Certificate Services](https://msdn.microsoft.com/en-us/library/ff630887.aspx).
Managing certificates in-house may be more complex and security risky, but
offers an alternative solution to partnering with a certificate provider.
### Legacy CAs
Some Enterprises rely on Certificate Authorities that have not been audited to
the same standard as other CAs or been operated to the same security
requirements. These CAs would not be trusted in new products, nor other root
programs, but may be trusted on one or more platforms that Chrome runs on.
Because they are trusted by default, they are subject to the Chrome's policies
on requiring CT, but due to their legacy status, may not be prepared. While the
requirement to disclose new certificates via Certificate Transparency has been
communicated, some may not do so, causing their new certificates to not be
trusted. This is most common with CAs run by governments, as they rarely meet the
required security standards of a widely-trusted CA.
Organizations that need to use certificates from these CAs should be aware
that their certificates will not be trusted if they do not support CT, and so
should look for CAs that do support CT. Alternatively, supporting CT via TLS
may be the only way to ensure these certificates continue to work, but that
requires the Enterprise constantly keep track of changes regarding Certificate
Transparency.
Organizations that need to trust certificates from these CAs, such as when
talking to other organizations that need to use these CAs, can configure
[Enterprise Policies](#Enterprise-Policy) for users in their organization,
which will allow trust in these certificates. As these only apply to Enterprise
users, these policies are not suitable for making these certificates trusted
more widely.
### Enterprise Policies
Several Chrome-specific policies exist that allow Enterprises to configure
their machines or users to disable Certificate Transparency for certain cases.
These policies are documented in the
[master policy list](https://cloud.google.com/docs/chrome-enterprise/policies),
but detailed further below.
#### CertificateTransparencyEnforcementDisabledForUrls
This [policy](https://cloud.google.com/docs/chrome-enterprise/policies/?policy=CertificateTransparencyEnforcementDisabledForUrls)
has been available since Chrome 53, and allows for disabling Certificate
Transparency enforcement for a certain set of domains or subdomains, without
disabling Certificate Transparency altogether.
If you wish to disable CT for a given hostname, and all of its subdomains, then
the domain is simply entered into the list. For example, `example.com` will
disable CT for `example.com` and all subdomains.
If you wish to disable CT only for a given hostname, but wish to ensure that
subdomains will still have CT enabled, then prefix the domain with a leading
dot. For example, `.example.com` will disable CT for `example.com` exactly,
while leaving it enabled for subdomains.
#### CertificateTransparencyEnforcementDisabledForCas
This [policy](https://cloud.google.com/docs/chrome-enterprise/policies/?policy=CertificateTransparencyEnforcementDisabledForCas),
available since Chrome 57, allows for disabling Certificate Transparency
enforcement if certain conditions are met in the trusted certificate chain.
This allows disabling CT without having to list all of the domain names, but
only for certificates issued to a specific organization.
Certificates are specified in this policy by applying Base64 to a hash of their
subjectPublicKeyInformation, as well as specifying the hash algorithm used.
This format is very similar to that used by
[HTTP Public Key Pinning](https://tools.ietf.org/html/rfc7469) (HPKP), so that
sites can use the same [examples](https://tools.ietf.org/html/rfc7469#appendix-A)
or [tools](https://report-uri.com/home/pubkey_hash) used to generate HPKP
hashes to determine how to configure the policy. Note that while both use
Base64, an HPKP hash will be in the form `pin-sha256="hash"`, while the policy
will be in the form `sha256/hash`.
To disable Certificate Transparency for these certificates, the certificate
must match one of the following conditions:
1. The hash specified is of the server certificate's subjectPublicKeyInfo.
2. The hash specified is of an intermediate CA, and that intermediate CA has
a nameConstraints extension with one or more directoryNames in the
permittedSubtrees of that extension.
3. The hash specified is of an intermediate CA, that intermediate CA contains
one or more organizationName (O) attribute in the subject, and the server
certificate's has the same number of organizationName attributes, with
byte-for-byte identical values, in the same exact order.
#### CertificateTransparencyEnforcementDisabledForLegacyCas
This [policy](https://cloud.google.com/docs/chrome-enterprise/policies/?policy=CertificateTransparencyEnforcementDisabledForLegacyCas),
available since Chrome 67, allows for disabling Certificate Transparency
enforcement for certain legacy CAs that have not adopted modern security and
audit requirements required of publicly-trusted CAs. This is particularly
tailored towards CAs that are trusted on some platforms that Chrome runs on,
but are not trusted on ChromeOS or Android, due to not meeting the necessary
security requirements.
CAs are specified in this policy by applying Base64 to a hash of their
subjectPublicKeyInformation, the same as in
[CertificateTransparencyEnforcementDisabledForCAs](#CertificateTransparencyEnforcementDisabledForCas).
However, these CAs must also be recognized as Legacy CAs in the
[`/net/data/ssl/root_stores/root_stores.json`](/net/data/ssl/root_stores/root_stores.json)
file, which means that they are not trusted on ChromeOS or Android, but are
trusted on another platform that Chrome runs on.
This policy is the riskiest of the three Enterprise policies, in that such
legacy CAs can represent the greatest security threat to an organization, as
they lack either the audits or compliance with industry best practice and root
store requirements. Enterprises should only enable this policy if no other
option meets their needs.
## Certificate Transparency for Chrome/Chromium developers
### //net Interfaces
Support for Certificate Transparency in //net is made up of two core
interfaces:
* [`CTVerifier`](/net/cert/ct_verifier.h): Responsible for extracting the
CT information (SCTs) from the certificate, the OCSP response, and the
TLS handshake, validating the signatures against a set of known/configured
CT logs, and validating that the SCTs match the certificate provided.
* [`CTPolicyEnforcer`](/net/cert/ct_policy_enforcer.h): Responsible for
taking the extracted, verified SCTs and applying
application/embedder-specific policies to determine whether the SCTs are
"good enough" (meet application requirements).
In addition to these two core classes, configuration and support for CT-related
behaviours is expressed via the
[`TransportSecurityState`](/net/http/transport_security_state.h). The
`TransportSecurityState` has methods for exposing support and policies for
[`Expect-CT`](https://tools.ietf.org/html/draft-ietf-httpbis-expect-ct) and
for embedder-specific overrides via the
`TransportSecurityState::RequireCTDelegate`.
### Supporting Certificate Transparency for Embedders
While Chromium has implemented support for Certificate Transparency for a
number of years, it would not block connections unless there was a known
security issue. For example, certificates that were intended to be EV, but
were not disclosed properly, simply would have their EV status removed, while
the connection should still continue.
However, as Google Chrome looks to roll out a more rigorous enforcement of
Certificate Transparency, by enforcing that newly-issued certificates are
disclosed as a condition of being trusted, the risks to the CA and CT
ecosystem significantly increase if embedders implement CT without the ability
for reliable, rapid updates, keeping track with ongoing development in the
main tree and reliably delivering security updates on the same cadence as
Chromium branches and Google Chrome releases.
For this reason, the CT implementation is undergoing a refactoring to reduce
those risks through code and implementation. As a result, Chromium embedders
will **NOT** have CT enforcement enabled by default, and are **NOT** encouraged
to manually enable it at this time.
Distributors of products that embed Chromium sources are encouraged to
participate in the
[ct-policy@chromium.org](https://groups.google.com/a/chromium.org/forum/#!forum/ct-policy)
discussion group, which involves a variety of stakeholders in the CT ecosystem
for discussing matters of policy and implementation, in order to understand
the risks and participate in solutions. Face-to-face summits are periodically
held to gather key stakeholders together to work through these issues, helping
root programs, CAs, log operators, and the overall PKI community develop
consistent, interoperable, secure, and reliable policies and implementations.

85
src/net/docs/certificate_lifetimes.md Normal file
View File

@@ -0,0 +1,85 @@
# Certificate Lifetimes
As part of our ongoing commitment to ensuring users’ security, Google is
reducing the maximum allowed lifetimes of TLS certificates.
## Upcoming Changes
Beginning with Chrome 85, TLS server certificates issued on or after
2020-09-01 00:00:00 UTC will be required to have a validity period of 398 days
or less. This will only apply to TLS server certificates from CAs that are
trusted in a default installation of Google Chrome, commonly known as
"publicly trusted CAs", and will not apply to locally-operated CAs that have
been manually configured.
Certificates that do not comply with this requirement will not work, and may
cause webpages to fail to load or to render incorrectly.
If a certificate that does not comply with this requirement is issued by a CA
trusted in a default installation of Google Chrome, this will be treated as a
failure to comply with the security policies necessary to being a trusted CA,
and may result in the removal of trust of that CA’s certificates.
## Technical Details
* A certificate will be impacted by this restriction if either the notBefore
of the certificate is on or after 2020-09-01 00:00:00 UTC, or if the first
precertificate logged by the CA to a Certificate Transparency Log that is
qualified at time of issuance is on or after this date.
* The validity period of a certificate is defined within RFC 5280, Section
4.1.2.5, as "the period of time from notBefore through notAfter, inclusive."
* 398 days is measured with a day being equal to 86,400 seconds. Any time
greater than this indicates an additional day of validity.
* To avoid the risk of misissuance, such as due to leap seconds or
CA-configured randomization, CAs SHOULD issue such server certificates with
validity periods of 397 days or less.
## Frequently Asked Questions
* Why is Chrome making this change?
* Shortening certificate lifetimes protects users by reducing the impact
of compromised keys, and by speeding up the replacement of insecure
technologies and practices across the web. Key compromises and the
discovery of internet security weaknesses are common events that can lead
to real-world harm, and the web’s users should be better protected against
them.
* Does this apply to locally-operated CAs, such as those used within
enterprises that use enterprise-configured configured CAs?
* No. This only applies to the set of CAs that are trusted by default by
Google Chrome, and not CAs that are operated by an enterprise and that
have no certification paths to CAs that are trusted by default.
* Is there an enterprise policy to disable this enforcement?
* No. These changes are transparent and do not offer an enterprise control
to override, as they only apply to so-called "publicly trusted" CAs.
Enterprises that wish to have certificates with validity periods longer
than 398 days may do so by using a locally-operated CA that does not have
any certification paths up to a publicly trusted CA.
* Does this mean I have to replace my existing certificates?
* No. This requirement only applies to new certificate issuance on or after
2020-09-01 00:00:00 UTC. Existing certificates whose validity period
exceeds 398 days will continue to work, while new certificates must comply
with these new requirements, such as when they are renewed or replaced.
* Will this make certificates more expensive?
* As with past changes to the maximum certificate lifetimes, many CAs have
committed to providing additional certificates, as needed by the shortened
maximum lifetime, at no additional cost.
* What will happen if a certificate is issued that does not meet these
requirements?
* Google Chrome will reject such certificates as having too long a validity
period, consistent with existing validity-period based enforcement.
Additionally, such certificates will be treated as a critical security
failure by the CA, and may result in further action taken on the CA that
may affect how current or future certificates from that CA function.
Chromium-based browsers will have this enforcement enabled by default, and
will need to modify the source to disable this.
* What are other browsers doing?
* Apple previously announced this change for versions of iOS, iPadOS, macOS,
tvOS, and watchOS, as documented at
https://support.apple.com/en-us/HT211025, which will apply to all
applications, and not just those of Safari. This certificate lifetime
requirement is fully interoperable with Apple’s requirements.
Microsoft, Mozilla, Opera, and 360 have previously indicated their support
for these requirements, although have not yet made announcements at the
time of this post (2020-06-22). Other browsers, including those browsers
based on Chromium, may provide additional guidance or clarification.

242
src/net/docs/code-patterns.md Normal file
View File

@@ -0,0 +1,242 @@
# Chrome Network Stack Common Coding Patterns
## Combined error and byte count into a single value
At many places in the network stack, functions return a value that, if
positive, indicate a count of bytes that the the function read or
wrote, and if negative, indicates a network stack error code (see
[net_error_list.h][]).
Zero indicates either `net::OK` or zero bytes read (usually EOF)
depending on the context. This pattern is generally specified by
an `int` return type.
Many functions also have variables (often named `result` or `rv`) containing
such a value; this is especially common in the [DoLoop](#DoLoop) pattern
described below.
## Sync/Async Return
Many network stack routines may return synchronously or
asynchronously. These functions generally return an int as described
above. There are three cases:
* If the value is positive or zero, that indicates a synchronous
successful return, with a zero return value indicating either zero
bytes/EOF or indicating `net::OK`, depending on context. If there
is a callback argument, it is not invoked.
* If the value is negative and != `net::ERR_IO_PENDING`, it is an error
code specifying a synchronous failure. If there is a callback argument,
it is not invoked.
* If the return value is the special value `net::ERR_IO_PENDING`, it
indicates that the routine will complete asynchronously. A reference to
any provided IOBuffer will be retained by the called entity until
completion, to be written into or read from as required.
If there is a callback argument, that callback will be called upon
completion with the return value; if there is no callback argument, it
usually means that some known callback mechanism will be employed.
## DoLoop
The DoLoop pattern is used in the network stack to construct simple
state machines. It is used for cases in which processing is basically
single-threaded and could be written in a single function, if that
function could block waiting for input. Generally, initiation of a
state machine is triggered by some method invocation by a class
consumer, and that state machine is driven (possibly across
asynchronous IO initiated by the class) until the operation requested
by the method invocation completes, at which point the state variable is
set to `STATE_NONE` and the consumer notified.
Cases which do not fit into this single-threaded, single consumer
operation model are generally adapted in some way to fit the model,
either by multiple state machines (e.g. independent state machines for
reading and writing, if each can be initiated while the other is
outstanding) or by storing information across consumer invocations and
returns that can be used to restart the state machine in the proper
state.
Any class using this pattern will contain an enum listing all states
of that machine, and define a function, `DoLoop()`, to drive that state
machine. If a class has multiple state machines (as above) it will
have multiple methods (e.g. `DoReadLoop()` and `DoWriteLoop()`) to drive
those different machines.
The characteristics of the DoLoop pattern are:
* Each state has a corresponding function which is called by `DoLoop()`
for handling when the state machine is in that state. Generally the
states are named STATE`_<`STATE_NAME`>` (upper case separated by
underscores), and the routine is named Do`<`StateName`>` (CamelCase).
For example:
enum State {
STATE_NONE,
STATE_INIT,
STATE_FOO,
STATE_FOO_COMPLETE,
};
int DoInit();
int DoFoo();
int DoFooComplete(int result);
* Each state handling function has two basic responsibilities in
addition to state specific handling: Setting the data member
(named `next_state_` or something similar)
to specify the next state, and returning a `net::Error` (or combined
error and byte count, as above).
* On each `DoLoop()` iteration, the function saves the next state to a local
variable and resets to a default state (`STATE_NONE`),
and then calls the appropriate state handling based on the
original value of the next state. This looks like:
do {
State state = io_state_;
next_state_ = STATE_NONE;
switch (state) {
case STATE_INIT:
result = DoInit();
break;
...
This pattern is followed primarily to ensure that in the event of
a bug where the next state isn't set, the loop terminates rather
than loops infinitely. It's not a perfect mitigation, but works
well as a defensive measure.
* If a given state may complete asynchronously (for example,
writing to an underlying transport socket), then there will often
be split states, such as `STATE_WRITE` and
`STATE_WRITE_COMPLETE`. The first state is responsible for
starting/continuing the original operation, while the second state
is responsible for handling completion (e.g. success vs error,
complete vs. incomplete writes), and determining the next state to
transition to.
* While the return value from each call is propagated through the loop
to the next state, it is expected that for most state transitions the
return value will be `net::OK`, and that an error return will also
set the state to `STATE_NONE` or fail to override the default
assignment to `STATE_DONE` to exit the loop and return that
error to the caller. This is often asserted with a DCHECK, e.g.
case STATE_FOO:
DCHECK_EQ(result, OK);
result = DoFoo();
break;
The exception to this pattern is split states, where an IO
operation has been dispatched, and the second state is handling
the result. In that case, the second state's function takes the
result code:
case STATE_FOO_COMPLETE:
result = DoFooComplete(result);
break;
* If the return value from the state handling function is
`net::ERR_IO_PENDING`, that indicates that the function has arranged
for `DoLoop()` to be called at some point in the future, when further
progress can be made on the state transitions. The `next_state_` variable
will have been set to the proper value for handling that incoming
call. In this case, `DoLoop()` will exit. This often occurs between
split states, as described above.
* The DoLoop mechanism is generally invoked in response to a consumer
calling one of its methods. While the operation that method
requested is occuring, the state machine stays active, possibly
over multiple asynchronous operations and state transitions. When
that operation is complete, the state machine transitions to
`STATE_NONE` (by a `DoLoop()` callee not setting `next_state_`) or
explicitly to `STATE_DONE` (indicating that the operation is
complete *and* the state machine is not amenable to further
driving). At this point the consumer is notified of the completion
of the operation (by synchronous return or asynchronous callback).
Note that this implies that when `DoLoop()` returns, one of two
things will be true:
* The return value will be `net::ERR_IO_PENDING`, indicating that the
caller should take no action and instead wait for asynchronous
notification.
* The state of the machine will be either `STATE_DONE` or `STATE_NONE`,
indicating that the operation that first initiated the `DoLoop()` has
completed.
This invariant reflects and enforces the single-threaded (though
possibly asynchronous) nature of the driven state machine--the
machine is always executing one requested operation.
* `DoLoop()` is called from two places: a) methods exposed to the consumer
for specific operations (e.g. `ReadHeaders()`), and b) an IO completion
callbacks called asynchronously by spawned IO operations.
In the first case, the return value from `DoLoop()` is returned directly
to the caller; if the operation completed synchronously, that will
contain the operation result, and if it completed asynchronously, it
will be `net::ERR_IO_PENDING`. For example (from
`HttpStreamParser`, abridged for clarity):
int HttpStreamParser::ReadResponseHeaders(
CompletionOnceCallback callback) {
DCHECK(io_state_ == STATE_NONE || io_state_ == STATE_DONE);
DCHECK(callback_.is_null());
DCHECK(!callback.is_null());
int result = OK;
io_state_ = STATE_READ_HEADERS;
result = DoLoop(result);
if (result == ERR_IO_PENDING)
callback_ = std::move(callback);
return result > 0 ? OK : result;
}
In the second case, the IO completion callback will examine the
return value from `DoLoop()`. If it is `net::ERR_IO_PENDING`, no
further action will be taken, and the IO completion callback will be
called again at some future point. If it is not
`net::ERR_IO_PENDING`, that is a signal that the operation has
completed, and the IO completion callback will call the appropriate
consumer callback to notify the consumer that the operation has
completed. Note that it is important that this callback be done
from the IO completion callback and not from `DoLoop()` or a
`DoLoop()` callee, both to support the sync/async error return
(DoLoop and its callees don't know the difference) and to avoid
consumer callbacks deleting the object out from under `DoLoop()`.
Example:
void HttpStreamParser::OnIOComplete(int result) {
result = DoLoop(result);
if (result != ERR_IO_PENDING && !callback_.is_null())
std::move(callback_).Run(result);
}
* The DoLoop pattern has no concept of different events arriving for
a single state; each state, if waiting, is waiting for one
particular event, and when `DoLoop()` is invoked when the machine is
in that state, it will handle that event. This reflects the
single-threaded model for operations spawned by the state machine.
Public class methods generally have very little processing, primarily wrapping
`DoLoop()`. For `DoLoop()` entry this involves setting the `next_state_`
variable, and possibly making copies of arguments into class members. For
`DoLoop()` exit, it involves inspecting the return and passing it back to
the caller, and in the asynchronous case, saving any passed completion callback
for executing by a future subsidiary IO completion (see above example).
This idiom allows synchronous and asynchronous logic to be written in
the same fashion; it's all just state transition handling. For mostly
linear state diagrams, the handling code can be very easy to
comprehend, as such code is usually written linearly (in different
handling functions) in the order it's executed.
For examples of this idiom, see
* [HttpStreamParser::DoLoop](https://source.chromium.org/chromium/chromium/src/+/HEAD:net/http/http_stream_parser.cc).
* [HttpNetworkTransaction::DoLoop](https://source.chromium.org/chromium/chromium/src/+/HEAD:net/http/http_network_transaction.cc)
[net_error_list.h]: https://chromium.googlesource.com/chromium/src/+/main/net/base/net_error_list.h#1

236
src/net/docs/crash-course-in-net-internals.md Normal file
View File

@@ -0,0 +1,236 @@
# A Crash Course in Debugging with chrome://net-internals
This document is intended to help get people started debugging network errors
with chrome://net-internals, with some commonly useful tips and tricks. This
document is aimed more at how to get started using some of its features to
investigate bug reports, rather than as a feature overview.
It would probably be useful to read
[life-of-a-url-request.md](life-of-a-url-request.md) before this document.
# What Data Net-Internals Contains
chrome://net-internals provides a view of browser activity from net/'s
perspective. For this reason, it lacks knowledge of tabs, navigation, frames,
resource types, etc.
The leftmost column presents a list of views. Most debugging is done with the
Events view, which will be all this document covers.
The top level network stack object is the URLRequestContext. The Events view
has information for all Chrome URLRequestContexts that are hooked up to the
single, global, NetLog object. This includes both incognito and
non-incognito profiles, among other things. The Events view only shows events
for the period that net-internals was open and running, and is incrementally
updated as events occur. The code attempts to add a top level event for
URLRequests that were active when the chrome://net-internals tab was opened, to
help debug hung requests, but that's best-effort only, and only includes
requests for the current profile and the system URLRequestContext.
The other views are all snapshots of the current state of the main
URLRequestContext's components, and are updated on a 5 second timer. These will
show objects that were created before chrome://net-internals was opened.
# Events vs Sources
The Events view shows events logged by the NetLog. The NetLog model is that
long-lived network stack objects, called sources, emit events over their
lifetime. A NetLogWithSource object contains a source ID, a NetLogSourceType,
and a pointer to the NetLog the source emits events to.
The Events view has a list of sources in a column adjacent to the list of views.
Sources that include an event with a net_error parameter with negative value
(that is, some kind of ERR_) are shown with red background. Sources whose
opening event has not ended yet are shown with white background. Other events
have green background. The search queries corresponding to the first two kinds
are `is:error` and `is:active`.
When one or more sources are selected, corresponding events show up in another
column to the right, sorted by source, and by time within each source. There
are two time values: t is measured from some reference point common to all
sources, and st is measured from the first event for each source. Time is
displayed in milliseconds.
Since the network stack is asynchronous, events from different sources will
often be interlaced in time, but Events view does not feature showing events from
different sources ordered by time. Large time gaps in the event list of a
single source usually mean that time is spent in the context of another source.
Some events come in pairs: a beginning and end event, between which other events
may occur. They are shown with + and - prefixes, respectively. The begin event
has a dt value which shows the duration. If the end event was captured, then
duration is calculated as the time difference between the begin and the end
events. Otherwise the time elapsed from the begin event until capturing
was stopped is displayed (a lower bound for actual duration), followed by a +
sign (for example, "dt=120+").
If there are no other events in between the begin and end, and the end event has
no parameters, then they are collapsed in a single line without a sign prefix.
Some other events only occur at a single point in time, and will not have either
a sign prefix, or a dt duration value.
Generally only one event can be occuring for a source at a time. If there can
be multiple events doing completely independent things, the code often uses new
sources to represent the parallelism.
Most, but not all events correspond to a source. Exceptions are global events,
which have no source, and show up as individual entries in the source list.
Examples of global events include NETWORK_CHANGED, DNS_CONFIG_CHANGED, and
PROXY_CONFIG_CHANGED.
# Common source types
"Sources" correspond to certain net objects, however, multiple layers of net/
will often log to a single source. Here are the main source types and what they
include (excluding HTTP2 [SPDY]/QUIC):
* URL_REQUEST: This corresponds to the URLRequest object. It includes events
from all the URLRequestJobs, HttpCache::Transactions, NetworkTransactions,
HttpStreamRequests, HttpStream implementations, and HttpStreamParsers used to
service a response. If the URL_REQUEST follows HTTP redirects, it will include
each redirect. This is a lot of stuff, but generally only one object is doing
work at a time. This event source includes the full URL and generally includes
the request / response headers (except when the cache handles the response).
* HTTP_STREAM_JOB: This corresponds to HttpStreamFactory::Job (note that one
Request can have multiple Jobs). It also includes its proxy and DNS lookups.
HTTP_STREAM_JOB log events are separate from URL_REQUEST because two stream
jobs may be created and races against each other, in some cases -- one for
QUIC, and one for HTTP.
One of the final events of this source, before the
HTTP_STREAM_JOB_BOUND_TO_REQUEST event, indicates how an HttpStream was
created:
+ A SOCKET_POOL_BOUND_TO_CONNECT_JOB event means that a new TCP socket was
created, whereas a SOCKET_POOL_REUSED_AN_EXISTING_SOCKET event indicates that
an existing TCP socket was reused for a non-HTTP/2 request.
+ An HTTP2_SESSION_POOL_IMPORTED_SESSION_FROM_SOCKET event indicates that a
new HTTP/2 session was opened by this Job.
+ An HTTP2_SESSION_POOL_FOUND_EXISTING_SESSION event indicates that the request
was served on a preexisting HTTP/2 session.
+ An HTTP2_SESSION_POOL_FOUND_EXISTING_SESSION_FROM_IP_POOL event means that
the request was pooled to a preexisting HTTP/2 session which had a different
SpdySessionKey, but DNS resolution resulted in the same IP, and the
certificate matches.
+ There are currently no events logged for opening new QUIC sessions or
reusing existing ones.
* \*_CONNECT_JOB: This corresponds to the ConnectJob subclasses that each socket
pool uses. A successful CONNECT_JOB returns a SOCKET. The events here vary a
lot by job type. Their main event is generally either to create a socket, or
request a socket from another socket pool (which creates another CONNECT_JOB)
and then do some extra work on top of that -- like establish an SSL connection on
top of a TCP connection.
* SOCKET: These correspond to TCPSockets, but may also have other classes
layered on top of them (like an SSLClientSocket). This is a bit different from
the other classes, where the name corresponds to the topmost class, instead of
the bottommost one. This is largely an artifact of the fact the socket is
created first, and then SSL (or a proxy connection) is layered on top of it.
SOCKETs may be reused between multiple requests, and a request may end up
getting a socket created for another request.
* HOST_RESOLVER_IMPL_JOB: These correspond to HostResolverImpl::Job. They
include information about how long the lookup was queued, each DNS request that
was attempted (with the platform or built-in resolver) and all the other sources
that are waiting on the job.
When one source depends on another, the code generally logs an event at both
sources with a `source_dependency` value pointing to the other source. These
are clickable in the UI, adding the referred source to the list of selected
sources.
# Debugging
When you receive a report from the user, the first thing you'll generally want
to do find the URL_REQUEST[s] that are misbehaving. If the user gives an ERR_*
code or the exact URL of the resource that won't load, you can just search for
it. If it's an upload, you can search for "post", or if it's a redirect issue,
you can search for "redirect". However, you often won't have much information
about the actual problem. There are two filters in net-internals that can help
in a lot of cases:
* "type:URL_REQUEST is:error" will restrict the source list to URL_REQUEST
objects with an error of some sort. Cache errors are often non-fatal, so you
should generally ignore those, and look for a more interesting one.
* "type:URL_REQUEST sort:duration" will show the longest-lived requests first.
This is often useful in finding hung or slow requests.
For a list of other filter commands, you can mouse over the question mark on
chrome://net-internals.
Once you locate the problematic request, the next is to figure out where the
problem is -- it's often one of the last events, though it could also be related
to response or request headers. You can use `source_dependency` links to
navigate between related sources. You can use the name of an event to search
for the code responsible for that event, and try to deduce what went wrong
before/after a particular event.
Some things to look for while debugging:
* CANCELLED events almost always come from outside the network stack.
* Changing networks and entering / exiting suspend mode can have all sorts of
fun and exciting effects on underway network activity. Network changes log a
top level NETWORK_CHANGED event. Suspend events are currently not logged.
* URL_REQUEST_DELEGATE_\* / NETWORK_DELEGATE_\* / DELEGATE_INFO events mean a
URL_REQUEST is blocked on a URLRequest::Delegate or the NetworkDelegate, which
are implemented outside the network stack. A request will sometimes be CANCELED
here for reasons known only to the delegate. Or the delegate may cause a hang.
In general, to debug issues related to delegates, one needs to figure out which
method of which object is causing the problem. The object may be the a
NetworkDelegate, a ResourceThrottle, a ResourceHandler, the ResourceLoader
itself, or the ResourceDispatcherHost.
* Sockets are often reused between requests. If a request is on a stale
(reused) socket, what was the previous request that used the socket, how long
ago was it made? (Look at SOCKET_IN_USE events, and the HTTP_STREAM_JOBS they
point to via the `source_dependency` value.)
* SSL negotation is a process fraught with peril, particularly with broken
proxies. These will generally stall or fail in the SSL_CONNECT phase at the
SOCKET layer.
* Range requests have magic to handle them at the cache layer, and are often
issued by the media and PDF code.
* Late binding: HTTP_STREAM_JOBs are not associated with any CONNECT_JOB until
a CONNECT_JOB actually connects. This is so the highest priority pending
HTTP_STREAM_JOB gets the first available socket (which may be a new socket, or
an old one that's freed up). For this reason, it can be a little tricky to
relate hung HTTP_STREAM_JOBs to CONNECT_JOBs.
* Each CONNECT_JOB belongs to a "group", which has a limit of 6 connections. If
all CONNECT_JOBs belonging to a group (the CONNECT_JOB's description field) are
stalled waiting on an available socket, the group probably has 6 sockets that
that are hung -- either hung trying to connect, or used by stalled requests and
thus outside the socket pool's control.
* There's a limit on number of DNS resolutions that can be started at once. If
everything is stalled while resolving DNS addresses, you've probably hit this
limit, and the DNS lookups are also misbehaving in some fashion.
# Miscellany
These are just miscellaneous things you may notice when looking through the
logs.
* Some HTTP requests are not handled by URLRequestHttpJobs. These include
things like HSTS redirects (URLRequestRedirectJob), ServiceWorker, etc. These
generally don't log as much information, so it can be tricky to figure out
what's going on with these.
* Non-HTTP requests also appear in the log, and also generally don't log much
(blob URLs, chrome URLs, etc).
* Preconnects create a "HTTP_STREAM_JOB" event that may create multiple
CONNECT_JOBs (or none) and is then destroyed. These can be identified by the
"SOCKET_POOL_CONNECTING_N_SOCKETS" events.

35
src/net/docs/filter.md Normal file
View File

@@ -0,0 +1,35 @@
The network stack implements support for Content-Encodings using
"source streams", which can be composed together and mutate all the incoming
bytes from a URLRequestJob. Currently, the following streams are implemented:
* gzip (handling "deflate" and "gzip" Content-Encodings)
* brotli (handling "br" Content-Encoding)
Source streams conceptually form a chain, with the URLRequestJob as both the
beginning and end of the chain, meaning the URLRequestJob produces raw bytes at
the end and consumes unencoded bytes at the beginning. For example, to support a
hypothetical "Content-Encoding: bar,foo", streams would be arranged like so,
with "X <-- Y" meaning "data flows from Y to X", "X reads data from Y", or
"X is downstream from Y".
URLRequestJob <-- BarSourceStream <-- FooSourceStream <-- URLRequestJob
(URLRequestSourceStream)
Here URLRequestJob pulls filtered bytes from its upstream, BarSourceStream,
which pulls filtered bytes from FooSourceStream, which in turn pulls raw bytes
from URLRequestJob. URLRequestSourceStream is the ultimate upstream, which
produces the raw data. Every stream in the chain owns its upstream. In this
case, the ultimate downstream, URLRequestJob, owns its upstream,
BarSourceStream, which in turn owns FooSourceStream. FooSourceStream owns its
upstream, URLRequestSourceStream.
All source streams conform to the following interface (named SourceStream in the
tree):
int Read(IOBuffer* dest_buffer, size_t buffer_size,
const OnReadCompleteCallback& callback);
This function can return either synchronously or asynchronously via the supplied
callback. The source stream chain is "pull-based", in that data does not
propagate through the chain until requested by the final consumer of the
filtered data.

24
src/net/docs/generate-dot-to-png.py Executable file
View File

@@ -0,0 +1,24 @@
#!/usr/bin/env python
# Copyright 2018 The Chromium Authors
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
"""The diagrams included in the network stack documentation were
generated with Graphviz, and both source (.dot) and output (.svg) are
included in the repository. If graphviz is installed, the output may
be regenerated by running this script."""
import glob
import os
import subprocess
def main():
for dot_filename in glob.glob("*.dot"):
png_filename = os.path.splitext(dot_filename)[0] + ".png"
print "Generating %s from %s" % (png_filename, dot_filename)
subprocess.check_call(["dot", "-Tpng", dot_filename, "-o", png_filename])
subprocess.check_call(["optipng", png_filename])
if __name__ == "__main__":
main()

327
src/net/docs/life-of-a-feature.md Normal file
View File

@@ -0,0 +1,327 @@
# Life of a Feature
In the years since the Chromium browser was first open-sourced, the `//net`
directory has expanded from being the basis of loading web content in the
Chromium browser to accommodating a wide variety of networking needs,
both in the Chromium browser and in other Google and third-party products
and projects.
This brings with it many new opportunities, such as the ability to
introduce new protocols rapidly or push Web security forward, as well as
new challenges, such as how to balance the needs of various `//net`
consumers effectively.
To make it easier to contribute new features or to change existing
behaviors in `//net`, this document tries to capture the life of a
feature in `//net`, from initial design to the eventual possibility of
deprecation and removal.
## Supported Projects
When considering the introduction of a new `//net` feature or changing
a `//net` behavior, it's first necessary to understand where `//net`
is used, how it is used, and what the various constraints and limits are.
To understand a more comprehensive matrix of the supported platforms and
constraints, see [Supported Projects](supported-projects.md). When
examining a new feature request, or a change in behavior, it's necessary
to consider dimensions such as:
* Does this feature apply to all supported projects, or is this something
like a Browser-only feature?
* Does this feature apply to all supported platforms, or is this something
specific to a particular subset?
* Is the feature a basic networking library feature, or is it specific to
something in the Web Platform?
* Will some projects wish to strip the feature in order to meet targets
such as memory usage (RAM) or binary size?
* Does it depend on Google services or Google-specific behaviors or
features?
* How will this feature be tested / experimented with? For example,
__Field Trials (Finch)__ and __User Metrics (UMA)__ may not be available
on a number of platforms.
* How risky is the feature towards compatibility/stability? How will it
be undone if there is a bug?
* Are the power/memory/CPU/bandwidth requirements appropriate for the
targeted projects and/or platforms?
## Design and Layering
Once the supported platforms and constraints are identified, it's necessary
to determine how to actually design the feature to meet those constraints,
in hopefully the easiest way possible both for implementation and consumption.
### Designing for multiple platforms
In general, `//net` features try to support all platforms with a common
interface, and generally eschew OS-specific interfaces from being exposed as
part of `//net`.
Cross-platform code is generally done via declaring an interface named
`foo.h`, which is common for all platforms, and then using the build-system to
do compile-time switching between implementations in `foo_win.cc`, `foo_mac.cc`,
`foo_android.cc`, etc.
The goal is to ensure that consumers generally don't have to think about
OS-specific considerations, and can instead code to the interface.
### Designing for multiple products
While premature abstraction can significantly harm readability, if it is
anticipated that different products will have different implementation needs,
or may wish to selectively disable the feature, it's often necessary to
abstract that interface sufficiently in `//net` to allow for dependency
injection.
This is true whether discussing concrete classes and interfaces or something
as simple a boolean configuration flag that different consumers wish to set
differently.
The two most common approaches in `//net` are injection and delegation.
#### Injection
Injection refers to the pattern of defining the interface or concrete
configuration parameter (such as a boolean), along with the concrete
implementation, but requiring the `//net` embedder to supply an instance
of the interface or the configuration parameters (perhaps optionally).
Examples of this pattern include things such as the `ProxyConfigService`,
which has concrete implementations in `//net` for a variety of platforms'
configuration of proxies, but which requires it be supplied as part of the
`URLRequestContextGetter` building, if proxies are going to be supported.
An example of injecting configuration flags can be seen in the
`HttpNetworkSessionParams` structure, which is used to provide much of
the initialization parameters for the HTTP layer.
The ideal form of injection is to pass ownership of the injected object,
such as via a `std::unique_ptr<Foo>`. While this is not consistently
applied in `//net`, as there are a number of places in which ownership is
either shared or left to the embedder, with the injected object passed
around as a naked/raw pointer, this is generally seen as an anti-pattern
and not to be mirrored for new features.
#### Delegation
Delegation refers to forcing the `//net` embedder to respond to specific
delegated calls via a Delegate interface that it implements. In general,
when using the delegate pattern, ownership of the delegate should be
transferred, so that the lifetime and threading semantics are clear and
unambiguous.
That is, for a given class `Foo`, which has a `Foo::Delegate` interface
defined to allow embedders to alter behavior, prefer a constructor that
is
```
explicit Foo(std::unique_ptr<Delegate> delegate);
```
so that it is clear that the lifetime of `delegate` is determined by
`Foo`.
While this may appear similar to Injection, the general difference
between the two approaches is determining where the bulk of the
implementation lies. With Injection, the interface describes a
behavior contract that concrete implementations must adhere to; this
allows for much more flexibility with behavior, but with the downside
of significantly more work to implement or extend. Delegation attempts
to keep the bulk of the implementation in `//net`, and the decision as
to which implementation to use in `//net`, but allows `//net` to
provide specific ways in which embedders can alter behaviors.
The most notable example of the delegate pattern is `URLRequest::Delegate`,
which keeps the vast majority of the loading logic within `URLRequest`,
but allows the `URLRequest::Delegate` to participate during specific times
in the request lifetime and alter specific behaviors as necessary. (Note:
`URLRequest::Delegate`, like much of the original `//net` code, doesn't
adhere to the recommended lifetime patterns of passing ownership of the
Delegate. It is from the experience debugging and supporting these APIs
that the `//net` team strongly encourages all new code pass explicit
ownership, to reduce the complexity and risk of lifetime issues).
While the use of a `base::RepeatingCallback` can also be considered a form of
delegation, the `//net` layer tries to eschew any callbacks that can be
called more than once, and instead favors defining class interfaces
with concrete behavioral requirements in order to ensure the correct
lifetimes of objects and to adjust over time. When `//net` takes a
callback (e.g. `net::CompletionOnceCallback`), the intended pattern is to
signal the asynchronous completion of a single method, invoking that
callback at most once before deallocating it. For more discussion
of these patterns, see [Code Patterns](code-patterns.md).
### Understanding the Layering
A significant challenge many feature proposals face is understanding the
layering in `//net` and what different portions of `//net` are allowed to
know.
#### Socket Pools
The most common challenge feature proposals encounter is the awareness
that the act of associating an actual request to make with a socket is
done lazily, referred to as "late-binding".
With late-bound sockets, a given `URLRequest` will not be assigned an actual
transport connection until the request is ready to be sent. This allows for
reprioritizing requests as they come in, to ensure that higher priority requests
get preferential treatment, but it also means that features or data associated
with a `URLRequest` generally don't participate in socket establishment or
maintenance.
For example, a feature that wants to associate the low-level network socket
with a `URLRequest` during connection establishment is not something that the
`//net` design supports, since the `URLRequest` is kept unaware of how sockets
are established by virtue of the socket pools and late binding. This allows for
more flexibility when working to improve performance, such as the ability to
coalesce multiple logical 'sockets' over a single HTTP/2 or QUIC stream, which
may only have a single physical network socket involved.
#### Making Additional Requests
From time to time, `//net` feature proposals will involve needing to load
a secondary resource as part of processing. For example, feature proposals have
involved fetching a `/.well-known/` URI or reporting errors to a particular URL.
This is particularly challenging, because often, these features are implemented
deeper in the network stack, such as [`//net/cert`](../cert), [`//net/http`](../http),
or [`//net/filter`](../filter), which [`//net/url_request`](../url_request) depends
on. Because `//net/url_request` depends on these low-level directories, it would
be a circular dependency to have these directories depend on `//net/url_request`,
and circular dependencies are forbidden.
The recommended solution to address this is to adopt the delegation or injection
patterns. The lower-level directory will define some interface that represents the
"I need this URL" request, and then elsewhere, in a directory allowed to depend
on `//net/url_request`, an implementation of that interface/delegate that uses
`//net/url_request` is implemented.
### Understanding the Lifetimes
Understanding the object lifetime and dependency graph can be one of the largest
challenges to contributing and maintaining `//net`. As a consequence, features
which require introducing more complexity to the lifetimes of objects generally
have a greater challenge to acceptance.
The `//net` stack is designed heavily around a sync-or-async pattern, as
documented in [Code Patterns](code-patterns.md), while also having a strong
requirement that it be possible to cleanly shutdown the network stack. As a
consequence, features should have precise, well-defined lifetime semantics
and support graceful cleanup. Further, because much of the network stack can
have web-observable side-effects, it is often required for tasks to have
defined sequencing that cannot be reordered. To be ensure these requirements
are met, features should attempt to model object lifetimes as a hierarchical
DAG, using explicit ownership and avoiding the use of reference counting or
weak pointers as part of any of the exposed API contracts (even for features
only consumed in `//net`). Features that pay close attention to the lifetime
semantics are more likely to be reviewed and accepted than those that leave
it ambiguous.
In addition to preferring explicit lifetimes, such as through judicious use of
`std::unique_ptr<>` to indicate ownership transfer of dependencies, many
features in `//net` also expect that if a `base::{Once, Repeating}Callback` is
involved (which includes `net::CompletionOnceCallback`), then it's possible that
invoking the callback may result in the deletion of the current (calling)
object. As further expanded upon in [Code Patterns](code-patterns.md), features
and changes should be designed such that any callback invocation is the last bit
of code executed, and that the callback is accessed via the stack (such as
through the use of `std::move(callback_).Run()`.
### Specs: What Are They Good For
As `//net` is used as the basis for a number of browsers, it's an important part
of the design philosophy to ensure behaviors are well-specified, and that the
implementation conforms to those specifications. This may be seen as burdensome
when it's unclear whether or not a feature will 'take off,' but it's equally
critical to ensure that the Chromium projects do not fork the Web Platform.
#### Incubation Is Required
`//net` respects Chromium's overall position of [incubation first](https://groups.google.com/a/chromium.org/d/msg/blink-dev/PJ_E04kcFb8/baiLN3DTBgAJ) standards development.
With an incubation first approach, before introducing any new features that
might be exposed over the wire to servers, whether they are explicit behaviors,
such as adding new headers, or implicit behaviors such as
[Happy Eyeballs](https://tools.ietf.org/html/rfc6555), should have some form
of specification written. That specification should at least be on an
incubation track, and the expectation is that the specification should have a
direct path to an appropriate standards track. Features which don't adhere to
this pattern, or which are not making progress towards a standards track, will
require high-level approvals, to ensure that the Platform doesn't fragment.
#### Introducing New Headers
A common form of feature request is the introduction of new headers, either via
the `//net` implementation directly, or through consuming `//net` interfaces
and modifying headers on the fly.
The introduction of any additional headers SHOULD have an incubated spec attached,
ideally with cross-vendor interest. Particularly, headers which only apply to
Google or Google services are very likely to be rejected outright.
#### Making Additional Requests
While it's necessary to provide abstraction around `//net/url_request` for
any lower-level components that may need to make additional requests, for most
features, that's not all that is necessary. Because `//net/url_request` only
provides a basic HTTP fetching mechanism, it's insufficient for any Web Platform
feature, because it doesn't consider the broader platform concerns such as
interactions with CORS, Service Workers, cookie and authentication policies, or
even basic interactions with optional features like Extensions or SafeBrowsing.
To account for all of these things, any resource fetching that is to support
a feature of the Web Platform, whether because the resource will be directly
exposed to web content (for example, an image load or prefetch) or because it
is in response to invoking a Web Platform API (for example, invoking the
credential management API), the feature's resource fetching should be
explainable in terms of the [Fetch Living Standard](https://fetch.spec.whatwg.org/).
The Fetch standard defines a JavaScript API for fetching resources, but more
importantly, defines a common set of infrastructure and terminology that
tries to define how all resource loads in the Web Platform happen - whether
it be through the JavaScript API, through `XMLHttpRequest`, or the `src`
attribute in HTML tags, for example.
This also includes any resource fetching that wishes to use the same socket
pools or caches as the Web Platform, to ensure that every resource that is
web exposed (directly or indirectly) is fetched in a consistent and
well-documented way, thus minimizing platform fragmentation and security
issues.
There are exceptions to this, however, but they're generally few and far
between. In general, any feature that needs to define an abstraction to
allow it to "fetch resources," likely needs to also be "explained in terms
of Fetch".
## Implementation
In general, prior to implementing, try to get a review on net-dev@chromium.org
for the general feedback and design review.
In addition to the net-dev@chromium.org early review, `//net` requires that any
browser-exposed behavior should also adhere to the
[Blink Process](https://www.chromium.org/blink#new-features), which includes an
"Intent to Implement" message to blink-dev@chromium.org
For features that are unclear about their future, such as experiments or trials,
it's also expected that the design planning will also account for how features
will be removed cleanly. For features that radically affect the architecture of
`//net`, expect a high bar of justification, since reversing those changes if
they fail to pan out can cause significant disruption to productivity and
stability.
## Deprecation
Plan for obsolence, hope for success. Similar to implementation, features that
are to be removed should also go through the
[Blink Process](https://www.chromium.org/blink#TOC-Web-Platform-Changes:-Process)
for removing features.
Note that due to the diversity of [Supported Projects](supported-projects.md),
removing a feature while minimizing disruption can be just as complex as adding
a feature. For example, relying solely on __User Metrics (UMA)__ to signal the
safety of removing a feature may not consider all projects, and relying on
__Field Trials (Finch)__ to assess risk or restore the 'legacy' behavior may not
work on all projects either.
It's precisely because of these challenges that there's such a high bar for
adding features, because they may represent multi-year commitments to support,
even when the feature itself is deprecated or targeted for removal.

619
src/net/docs/life-of-a-url-request.md Normal file
View File

@@ -0,0 +1,619 @@
# Life of a URLRequest
This document gives an overview of the browser's lower-layers for networking.
Networking in the browser ranges from high level Javascript APIs like
`fetch()`, all the way down to writing encrypted bytes on a socket.
This document assumes that requests for URLs are mediated through the browser's
[Network Service](../../services/network/README.md), and focuses on all the
layers below the Network Service, including key points of integration.
It's particularly targeted at people new to the Chrome network stack, but
should also be useful for team members who may be experts at some parts of the
stack, but are largely unfamiliar with other components. It starts by walking
through how a basic request issued by another process works its way through the
network stack, and then moves on to discuss how various components plug in.
If you notice any inaccuracies in this document, or feel that things could be
better explained, please do not hesitate to submit patches.
# Anatomy of the Network Stack
The network stack is located in //net/ in the Chrome repo, and uses the
namespace "net". Whenever a class name in this doc has no namespace, it can
generally be assumed it's in //net/ and is in the net namespace.
The top-level network stack object is the URLRequestContext. The context has
non-owning pointers to everything needed to create and issue a URLRequest. The
context must outlive all requests that use it. Creating a context is a rather
complicated process usually managed by URLRequestContextBuilder.
The primary use of the URLRequestContext is to create URLRequest objects using
URLRequestContext::CreateRequest(). The URLRequest is the main interface used
by direct consumers of the network stack. It manages loading URLs with the
http, https, ws, and wss schemes. URLs for other schemes, such as file,
filesystem, blob, chrome, and data, are managed completely outside of //net.
Each URLRequest tracks a single request across all redirects until an error
occurs, it's canceled, or a final response is received, with a (possibly empty)
body.
The HttpNetworkSession is another major network stack object. It owns the
HttpStreamFactory, the socket pools, and the HTTP/2 and QUIC session pools. It
also has non-owning pointers to the network stack objects that more directly
deal with sockets.
This document does not mention either of these objects much, but at layers
above the HttpStreamFactory, objects often grab their dependencies from the
URLRequestContext, while the HttpStreamFactory and layers below it generally
get their dependencies from the HttpNetworkSession.
# How many "Delegates"?
A URLRequest informs the consumer of important events for a request using two
main interfaces: the URLRequest::Delegate interface and the NetworkDelegate
interface.
The URLRequest::Delegate interface consists of a small set of callbacks needed
to let the embedder drive a request forward. The NetworkDelegate is an object
pointed to by the URLRequestContext and shared by all requests, and includes
callbacks corresponding to most of the URLRequest::Delegate's callbacks, as
well as an assortment of other methods.
# The Network Service and Mojo
The network service, which lives in //services/network/, wraps //net/ objects,
and provides cross-process network APIs and their implementations for the rest
of Chrome. The network service uses the namespace "network" for all its classes.
The Mojo interfaces it provides are in the network::mojom namespace. Mojo is
Chrome's IPC layer. Generally there's a `mojo::Remote<network::mojom::Foo>`
proxy object in the consumer's process which also implements
the network::mojom::Foo interface. When the proxy object's methods are invoked,
it passes the call and all its arguments over a Mojo IPC channel, using a
`mojo::Receiver<network::mojom::Foo>`, to an implementation of the
network::mojom::Foo interface in the network service (the implementation is
typically a class named network::Foo), which may be running in another process,
another thread in the consumer's process, or even the same thread in the
consumer's process.
The network::NetworkService object is singleton that is used by Chrome to create
all other network service objects. The primary objects it is used to create are
the network::NetworkContexts, each of which owns its own mostly independent
URLRequestContext. Chrome has a number of different NetworkContexts, as there
is often a need to keep cookies, caches, and socket pools separate for different
types of requests, depending on what's making the request. Here are the main
NetworkContexts used by Chrome:
* The system NetworkContext, created and owned by Chrome's
SystemNetworkContextManager, is used for requests that aren't associated with
particular user or Profile. It has no on-disk storage, so loses all state, like
cookies, after each browser restart. It has no in-memory http cache, either.
SystemNetworkContextManager also sets up global network service preferences.
* Each Chrome Profile, including incognito Profiles, has its own NetworkContext.
Except for incognito and guest profiles, these contexts store information in
their own on-disk store, which includes cookies and an HTTP cache, among other
things. Each of these NetworkContexts is owned by a StoragePartition object in
the browser process, and created by a Profile's ProfileNetworkContextService.
* On platforms that support apps, each Profile has a NetworkContext for each app
installed on that Profile. As with the main NetworkContext, these may have
on-disk data, depending on the Profile and the App.
# Life of a Simple URLRequest
A request for data is dispatched from some process, which results in creating
a network::URLLoader in the network service (which, on desktop platform, is
typically in its own process). The URLLoader then creates a URLRequest to
drive the network request. That job first checks the HTTP cache, and then
creates a network transaction object, if necessary, to actually fetch the data.
That transaction tries to reuse a connection if available. If none is available,
it creates a new one. Once it has established a connection, the HTTP request is
dispatched, the response read and parsed, and the result returned back up the
stack and sent over to the caller.
Of course, it's not quite that simple :-}.
Consider a simple request issued by some process other than the network
service's process. Suppose it's an HTTP request, the response is uncompressed,
no matching entry in the cache, and there are no idle sockets connected to the
server in the socket pool.
Continuing with a "simple" URLRequest, here's a bit more detail on how things
work.
### Request starts in some (non-network) process
Summary:
* In the browser process, the network::mojom::NetworkContext interface is used
to create a network::mojom::URLLoaderFactory.
* A consumer (e.g. the content::ResourceDispatcher for Blink, the
content::NavigationURLLoaderImpl for frame navigations, or a
network::SimpleURLLoader) passes a network::ResourceRequest object and
network::mojom::URLLoaderClient Mojo channel to the
network::mojom::URLLoaderFactory, and tells it to create and start a
network::mojom::URLLoader.
* Mojo sends the network::ResourceRequest over an IPC pipe to a
network::URLLoaderFactory in the network process.
Chrome has a single browser process which handles starting and configuring other
processes, tab management, and navigation, among other things, and multiple
child processes, which are generally sandboxed and have no network access
themselves, apart from the network service (Which either runs in its own
process, or potentially in the browser process to conserve RAM). There are
multiple types of child processes (renderer, GPU, plugin, network, etc). The
renderer processes are the ones that layout webpages and run HTML.
The browser process creates the top level network::mojom::NetworkContext
objects. The NetworkContext interface is privileged and can only be accessed
from the browser process. The browser process uses it to create
network::mojom::URLLoaderFactories, which can then be passed to less
privileged processes to allow them to load resources using the NetworkContext.
To create a URLLoaderFactory, a network::mojom::URLLoaderFactoryParams object
is passed to the NetworkContext to configure fields that other processes are
not trusted to set, for security and privacy reasons.
One such field is the net::IsolationInfo field, which includes:
* A net::NetworkIsolationKey, which is used to enforce the
[privacy sandbox](https://www.chromium.org/Home/chromium-privacy/privacy-sandbox)
in the network stack, separating network resources used by different sites in
order to protect against tracking a user across sites.
* A net::SiteForCookies, which is used to determine which site to send SameSite
cookies for. SameSite cookies prevent cross-site attacks by only being
accessible when that site is the top-level site.
* How to update these values across redirects.
A consumer, either in the browser process or a child process, that wants to
make a network request gets a URLLoaderFactory from the browser process through
some manner, assembles a bunch of parameters in the large
network::ResourceRequest object, creates a network::mojom::URLLoaderClient Mojo
channel for the network::mojom::URLLoader to use to talk back to it, and then
passes them all to the URLLoaderFactory, which returns a URLLoader object that
it can use to manage the network request.
### network::URLLoaderFactory sets up the request in the network service
Summary:
* network::URLLoaderFactory creates a network::URLLoader.
* network::URLLoader uses the network::NetworkContext's URLRequestContext to
create and start a URLRequest.
The network::URLLoaderFactory, along with all NetworkContexts and most of the
network stack, lives on a single thread in the network service. It gets a
reconstituted ResourceRequest object from the network::mojom::URLLoaderFactory
Mojo pipe, does some checks to make sure it can service the request, and if so,
creates a URLLoader, passing the request and the NetworkContext associated with
the URLLoaderFactory.
The URLLoader then calls into the NetworkContext's net::URLRequestContext to
create the URLRequest. The URLRequestContext has pointers to all the network
stack objects needed to issue the request over the network, such as the cache,
cookie store, and host resolver. The URLLoader then calls into the
network::ResourceScheduler, which may delay starting the request, based on
priority and other activity. Eventually, the ResourceScheduler starts the
request.
### Check the cache, request an HttpStream
Summary:
* The URLRequest asks the URLRequestJobFactory to create a URLRequestJob,
and gets a URLRequestHttpJob.
* The URLRequestHttpJob asks the HttpCache to create an HttpTransaction, and
gets an HttpCache::Transaction, assuming caching is enabled.
* The HttpCache::Transaction sees there's no cache entry for the request,
and creates an HttpNetworkTransaction.
* The HttpNetworkTransaction calls into the HttpStreamFactory to request an
HttpStream.
The URLRequest then calls into the URLRequestJobFactory to create a
URLRequestHttpJob, a subclass of URLRequestJob, and then starts it
(historically, non-network URL schemes were also disptched through the
network stack, so there were a variety of job types.) The
URLRequestHttpJob attaches cookies to the request, if needed. Whether or
not SameSite cookies are attached depends on the IsolationInfo's
SiteForCookies, the URL, and the URLRequest's request_initiator field.
The URLRequestHttpJob calls into the HttpCache to create an
HttpCache::Transaction. The cache checks for an entry with the same URL
and NetworkIsolationKey. If there's no matching entry, the
HttpCache::Transaction will call into the HttpNetworkLayer to create an
HttpNetworkTransaction, and transparently wrap it. The HttpNetworkTransaction
then calls into the HttpStreamFactory to request an HttpStream to the server.
### Create an HttpStream
Summary:
* HttpStreamFactory creates an HttpStreamFactory::Job.
* HttpStreamFactory::Job calls into the TransportClientSocketPool to
populate an ClientSocketHandle.
* TransportClientSocketPool has no idle sockets, so it creates a
TransportConnectJob and starts it.
* TransportConnectJob creates a StreamSocket and establishes a connection.
* TransportClientSocketPool puts the StreamSocket in the ClientSocketHandle,
and calls into HttpStreamFactory::Job.
* HttpStreamFactory::Job creates an HttpBasicStream, which takes
ownership of the ClientSocketHandle.
* It returns the HttpBasicStream to the HttpNetworkTransaction.
The HttpStreamFactory::Job creates a ClientSocketHandle to hold a socket,
once connected, and passes it into the ClientSocketPoolManager. The
ClientSocketPoolManager assembles the TransportSocketParams needed to
establish the connection and creates a group name ("host:port") used to
identify sockets that can be used interchangeably.
The ClientSocketPoolManager directs the request to the
TransportClientSocketPool, since there's no proxy and it's an HTTP request. The
request is forwarded to the pool's ClientSocketPoolBase<TransportSocketParams>'s
ClientSocketPoolBaseHelper. If there isn't already an idle connection, and there
are available socket slots, the ClientSocketPoolBaseHelper will create a new
TransportConnectJob using the aforementioned params object. This Job will do the
actual DNS lookup by calling into the HostResolverImpl, if needed, and then
finally establishes a connection.
Once the socket is connected, ownership of the socket is passed to the
ClientSocketHandle. The HttpStreamFactory::Job is then informed the
connection attempt succeeded, and it then creates an HttpBasicStream, which
takes ownership of the ClientSocketHandle. It then passes ownership of the
HttpBasicStream back to the HttpNetworkTransaction.
### Send request and read the response headers
Summary:
* HttpNetworkTransaction gives the request headers to the HttpBasicStream,
and tells it to start the request.
* HttpBasicStream sends the request, and waits for the response.
* The HttpBasicStream sends the response headers back to the
HttpNetworkTransaction.
* The response headers are sent up through the URLRequest, to the
network::URLLoader.
* They're then sent to the network::mojom::URLLoaderClient via Mojo.
The HttpNetworkTransaction passes the request headers to the HttpBasicStream,
which uses an HttpStreamParser to (finally) format the request headers and body
(if present) and send them to the server.
The HttpStreamParser waits to receive the response and then parses the HTTP/1.x
response headers, and then passes them up through both the
HttpNetworkTransaction and HttpCache::Transaction to the URLRequestHttpJob. The
URLRequestHttpJob saves any cookies, if needed, and then passes the headers up
to the URLRequest and on to the network::URLLoader, which sends the data over
a Mojo pipe to the network::mojom::URLLoaderClient, passed in to the URLLoader
when it was created.
### Response body is read
Summary:
* network::URLLoader creates a raw Mojo data pipe, and passes one end to the
network::mojom::URLLoaderClient.
* The URLLoader requests shared memory buffer from the Mojo data pipe.
* The URLLoader tells the URLRequest to write to the memory buffer, and tells
the pipe when data has been written to the buffer.
* The last two steps repeat until the request is complete.
Without waiting to hear back from the network::mojom::URLLoaderClient, the
network::URLLoader allocates a raw mojo data pipe, and passes the client the
read end of the pipe. The URLLoader then grabs an IPC buffer from the pipe,
and passes a 64KB body read request down through the URLRequest all the way
down to the HttpStreamParser. Once some data is read, possibly less than 64KB,
the number of bytes read makes its way back to the URLLoader, which then tells
the Mojo pipe the read was complete, and then requests another buffer from the
pipe, to continue writing data to. The pipe may apply back pressure, to limit
the amount of unconsumed data that can be in shared memory buffers at once.
This process repeats until the response body is completely read.
### URLRequest is destroyed
Summary:
* When complete, the network::URLLoaderFactory deletes the network::URLLoader,
which deletes the URLRequest.
* During destruction, the HttpNetworkTransaction determines if the socket is
reusable, and if so, tells the HttpBasicStream to return it to the socket pool.
When the URLRequest informs the network::URLLoader the request is complete, the
URLLoader passes the message along to the network::mojom::URLLoaderClient, over
its Mojo pipe, before telling the URLLoaderFactory to destroy the URLLoader,
which results in destroying the URLRequest and closing all Mojo pipes related to
the request.
When the HttpNetworkTransaction is being torn down, it figures out if the
socket is reusable. If not, it tells the HttpBasicStream to close the socket.
Either way, the ClientSocketHandle returns the socket is then returned to the
socket pool, either for reuse or so the socket pool knows it has another free
socket slot.
### Object Relationships and Ownership
A sample of the object relationships involved in the above process is
diagramed here:
![Object Relationship Diagram for URLRequest lifetime](url_request.png)
There are a couple of points in the above diagram that do not come
clear visually:
* The method that generates the filter chain that is hung off the
URLRequestJob is declared on URLRequestJob, but the only current
implementation of it is on URLRequestHttpJob, so the generation is
shown as happening from that class.
* HttpTransactions of different types are layered; i.e. a
HttpCache::Transaction contains a pointer to an HttpTransaction, but
that pointed-to HttpTransaction generally is an
HttpNetworkTransaction.
# Additional Topics
## HTTP Cache
The HttpCache::Transaction sits between the URLRequestHttpJob and the
HttpNetworkTransaction, and implements the HttpTransaction interface, just like
the HttpNetworkTransaction. The HttpCache::Transaction checks if a request can
be served out of the cache. If a request needs to be revalidated, it handles
sending a conditional revalidation request over the network. It may also break a
range request into multiple cached and non-cached contiguous chunks, and may
issue multiple network requests for a single range URLRequest.
The HttpCache::Transaction uses one of three disk_cache::Backends to actually
store the cache's index and files: The in memory backend, the blockfile cache
backend, and the simple cache backend. The first is used in incognito. The
latter two are both stored on disk, and are used on different platforms.
One important detail is that it has a read/write lock for each URL. The lock
technically allows multiple reads at once, but since an HttpCache::Transaction
always grabs the lock for writing and reading before downgrading it to a read
only lock, all requests for the same URL are effectively done serially. The
renderer process merges requests for the same URL in many cases, which mitigates
this problem to some extent.
It's also worth noting that each renderer process also has its own in-memory
cache, which has no relation to the cache implemented in net/, which lives in
the network service.
## Cancellation
A consumer can cancel a request at any time by deleting the
network::mojom::URLLoader pipe used by the request. This will cause the
network::URLLoader to destroy itself and its URLRequest.
When an HttpNetworkTransaction for a cancelled request is being torn down, it
figures out if the socket the HttpStream owns can potentially be reused, based
on the protocol (HTTP / HTTP/2 / QUIC) and any received headers. If the socket
potentially can be reused, an HttpResponseBodyDrainer is created to try and
read any remaining body bytes of the HttpStream, if any, before returning the
socket to the SocketPool. If this takes too long, or there's an error, the
socket is closed instead. Since this all happens at the layer below the cache,
any drained bytes are not written to the cache, and as far as the cache layer is
concerned, it only has a partial response.
## Redirects
The URLRequestHttpJob checks if headers indicate a redirect when it receives
them from the next layer down (typically the HttpCache::Transaction). If they
indicate a redirect, it tells the cache the response is complete, ignoring the
body, so the cache only has the headers. The cache then treats it as a complete
entry, even if the headers indicated there will be a body.
The URLRequestHttpJob then checks with the URLRequest if the redirect should be
followed. The URLRequest then informs the network::URLLoader about the redirect,
which passes information about the redirect to network::mojom::URLLoaderClient,
in the consumer process. Whatever issued the original request then checks
if the redirect should be followed.
If the redirect should be followed, the URLLoaderClient calls back into the
URLLoader over the network::mojom::URLLoader Mojo interface, which tells the
URLRequest to follow the redirect. The URLRequest then creates a new
URLRequestJob to send the new request. If the URLLoaderClient chooses to
cancel the request instead, it can delete the network::mojom::URLLoader
pipe, just like the cancellation case discussed above. In either case, the
old HttpTransaction is destroyed, and the HttpNetworkTransaction attempts to
drain the socket for reuse, as discussed in the previous section.
In some cases, the consumer may choose to handle a redirect itself, like
passing off the redirect to a ServiceWorker. In this case, the consumer cancels
the request and then calls into some other network::mojom::URLLoaderFactory
with the new URL to continue the request.
## Filters (gzip, deflate, brotli, etc)
When the URLRequestHttpJob receives headers, it sends a list of all
Content-Encoding values to Filter::Factory, which creates a (possibly empty)
chain of filters. As body bytes are received, they're passed through the
filters at the URLRequestJob layer and the decoded bytes are passed back to the
URLRequest::Delegate.
Since this is done above the cache layer, the cache stores the responses prior
to decompression. As a result, if files aren't compressed over the wire, they
aren't compressed in the cache, either.
## Socket Pools
The ClientSocketPoolManager is responsible for assembling the parameters needed
to connect a socket, and then sending the request to the right socket pool.
Each socket request sent to a socket pool comes with a socket params object, a
ClientSocketHandle, and a "group name". The params object contains all the
information a ConnectJob needs to create a connection of a given type, and
different types of socket pools take different params types. The
ClientSocketHandle will take temporary ownership of a connected socket and
return it to the socket pool when done. All connections with the same group name
in the same pool can be used to service the same connection requests, so it
consists of host, port, protocol, and whether "privacy mode" is enabled for
sockets in the goup.
All socket pool classes derive from the ClientSocketPoolBase<SocketParamType>.
The ClientSocketPoolBase handles managing sockets - which requests to create
sockets for, which requests get connected sockets first, which sockets belong
to which groups, connection limits per group, keeping track of and closing idle
sockets, etc. Each ClientSocketPoolBase subclass has its own ConnectJob type,
which establishes a connection using the socket params, before the pool hands
out the connected socket.
### Socket Pool Layering
Some socket pools are layered on top other socket pools. This is done when a
"socket" in a higher layer needs to establish a connection in a lower level
pool and then take ownership of it as part of its connection process. For
example, each socket in the SSLClientSocketPool is layered on top of a socket
in the TransportClientSocketPool. There are a couple additional complexities
here.
From the perspective of the lower layer pool, all of its sockets that a higher
layer pools owns are actively in use, even when the higher layer pool considers
them idle. As a result, when a lower layer pool is at its connection limit and
needs to make a new connection, it will ask any higher layer pools to close an
idle connection if they have one, so it can make a new connection.
Since sockets in the higher layer pool are also in a group in the lower layer
pool, they must have their own distinct group name. This is needed so that, for
instance, SSL and HTTP connections won't be grouped together in the
TcpClientSocketPool, which the SSLClientSocketPool sits on top of.
### Socket Pool Class Relationships
The relationships between the important classes in the socket pools is
shown diagrammatically for the lowest layer socket pool
(TransportSocketPool) below.
![Object Relationship Diagram for Socket Pools](pools.png)
The ClientSocketPoolBase is a template class templatized on the class
containing the parameters for the appropriate type of socket (in this
case TransportSocketParams). It contains a pointer to the
ClientSocketPoolBaseHelper, which contains all the type-independent
machinery of the socket pool.
When socket pools are initialized, they in turn initialize their
templatized ClientSocketPoolBase member with an object with which it
should create connect jobs. That object must derive from
ClientSocketPoolBase::ConnectJobFactory templatized by the same type
as the ClientSocketPoolBase. (In the case of the diagram above, that
object is a TransportConnectJobFactory, which derives from
ClientSocketPoolBase::ConnectJobFactory&lt;TransportSocketParams&gt;.)
Internally, that object is wrapped in a type-unsafe wrapper
(ClientSocketPoolBase::ConnectJobFactoryAdaptor) so that it can be
passed to the initialization of the ClientSocketPoolBaseHelper. This
allows the helper to create connect jobs while preserving a type-safe
API to the initialization of the socket pool.
### SSL
When an SSL connection is needed, the ClientSocketPoolManager assembles the
parameters needed both to connect the TCP socket and establish an SSL
connection. It then passes them to the SSLClientSocketPool, which creates
an SSLConnectJob using them. The SSLConnectJob's first step is to call into the
TransportSocketPool to establish a TCP connection.
Once a connection is established by the lower layered pool, the SSLConnectJob
then starts SSL negotiation. Once that's done, the SSL socket is passed back to
the HttpStreamFactory::Job that initiated the request, and things proceed
just as with HTTP. When complete, the socket is returned to the
SSLClientSocketPool.
## Proxies
Each proxy has its own completely independent set of socket pools. They have
their own exclusive TransportSocketPool, their own protocol-specific pool above
it, and their own SSLSocketPool above that. HTTPS proxies also have a second
SSLSocketPool between the the HttpProxyClientSocketPool and the
TransportSocketPool, since they can talk SSL to both the proxy and the
destination server, layered on top of each other.
The first step the HttpStreamFactory::Job performs, just before calling
into the ClientSocketPoolManager to create a socket, is to pass the URL to the
Proxy service to get an ordered list of proxies (if any) that should be tried
for that URL. Then when the ClientSocketPoolManager tries to get a socket for
the Job, it uses that list of proxies to direct the request to the right socket
pool.
## Alternate Protocols
### HTTP/2 (Formerly SPDY)
HTTP/2 negotation is performed as part of the SSL handshake, so when
HttpStreamFactory::Job gets a socket, it may have HTTP/2 negotiated over it
as well. When it gets a socket with HTTP/2 negotiated as well, the Job creates a
SpdySession using the socket and a SpdyHttpStream on top of the SpdySession.
The SpdyHttpStream will be passed to the HttpNetworkTransaction, which drives
the stream as usual.
The SpdySession will be shared with other Jobs connecting to the same server,
and future Jobs will find the SpdySession before they try to create a
connection. HttpServerProperties also tracks which servers supported HTTP/2 when
we last talked to them. We only try to establish a single connection to servers
we think speak HTTP/2 when multiple HttpStreamFactory::Jobs are trying to
connect to them, to avoid wasting resources.
### QUIC
QUIC works quite a bit differently from HTTP/2. Servers advertise QUIC support
with an "Alternate-Protocol" HTTP header in their responses.
HttpServerProperties then tracks servers that have advertised QUIC support.
When a new request comes in to HttpStreamFactory for a connection to a
server that has advertised QUIC support in the past, it will create a second
HttpStreamFactory::Job for QUIC, which returns an QuicHttpStream on success.
The two Jobs (one for QUIC, one for all versions of HTTP) will be raced against
each other, and whichever successfully creates an HttpStream first will be used.
As with HTTP/2, once a QUIC connection is established, it will be shared with
other Jobs connecting to the same server, and future Jobs will just reuse the
existing QUIC session.
## Prioritization
URLRequests are assigned a priority on creation. It only comes into play in
a couple places:
* The ResourceScheduler lives outside net/, and in some cases, delays starting
low priority requests on a per-tab basis.
* DNS lookups are initiated based on the highest priority request for a lookup.
* Socket pools hand out and create sockets based on prioritization. However,
when a socket becomes idle, it will be assigned to the highest priority request
for the server it's connected to, even if there's a higher priority request to
another server that's waiting on a free socket slot.
* HTTP/2 and QUIC both support sending priorities over-the-wire.
At the socket pool layer, sockets are only assigned to socket requests once the
socket is connected and SSL is negotiated, if needed. This is done so that if
a higher priority request for a group reaches the socket pool before a
connection is established, the first usable connection goes to the highest
priority socket request.
## Non-HTTP Schemes
WebSockets requests (wss:// and ws://) start as HTTP requests with an HTTP
upgrade header. Once the handshake completes successfully, the connection
is used as a full-duplex communication channel to the server for WebSocket
frames, rather than to receive an HTTP response body. WebSockets have their
own Mojo interfaces and //net classes, but internally they reuse the full
URLRequest machinery up to the point headers are received from the server.
Then the connection is handed off to the WebSocket code to manage.
Other schemes typically have their own network::mojom::URLLoaderFactory that
is not part of the network service. Standard schemes like file:// and blob://
are handled by the content layer and its dependencies
(content::FileURLLoaderFactory and storage::BlobURLLoaderFactory, respectively,
for those two schemes). Chrome-specific schemes, like externalfile:// and
chrome-extension:// are often handled by a URLLoaderFactory in the chrome layer,
though chrome:// itself is actually handled in //content.
data:// URLs are handled a bit differently from other schemes. If a renderer
process requests a data:// subresource, the renderer typically decodes it
internally, as sending it to an out-of-process URLLoader would be inefficient.
Navigations are a bit different. To navigate to a URL, the browser process
creates a URLLoader and passes it over to a renderer process. So in the
case of a navigation to a data:// URL, a URLLoader is created using a
content::DataURLLoaderFactory that lives in the browser process, and then a
mojo::Remote for the browser-hosted URLLoader is passed to a renderer
proceess.
about:blank is similarly often handled in the renderer, though there is a
factory for that used in the case of navigations as well. Other about: URLs
are mapped to the corresponding Chrome URLs by the navigation code, rather
than having that logic live in a URLLoaderFactory.

176
src/net/docs/net-log.md Normal file
View File

@@ -0,0 +1,176 @@
# NetLog
This document describes the design and use of logging through NetLog.
[TOC]
## Adding new NetLogging code
Adding information to the NetLog helps debugging. However, logging also requires
careful review as it can impact performance, privacy, and security.
Please add a [net/log/OWNERS](../log/OWNERS) reviewer when adding new NetLog
parameters, or adding information to existing ones.
The high level objectives when adding net logging code are:
* No performance cost when capturing is off.
* Logs captured using [`kDefault`](../log/net_log_capture_mode.h) are safe to
upload and share publicly.
* Capturing using [`kDefault`](../log/net_log_capture_mode.h) has a low
performance impact.
* Logs captured using [`kDefault`](../log/net_log_capture_mode.h) are small
enough to upload to bug reports.
* Events that may emit sensitive information have accompanying unit-tests.
* The event and its possible parameters are documented in
[net_log_event_type_list.h](../log/net_log_event_type_list.h)
To avoid doing work when logging is off, logging code should generally be
conditional on `NetLog::IsCapturing()`. Note that when specifying parameters
via a lambda, the lambda is already conditional on `IsCapturing()`.
### Binary data and strings
NetLog parameters are specified as a JSON serializable `base::Value`. This has
some subtle implications:
* Do not use `base::Value::Type::STRING` with non-UTF-8 data.
* Do not use `base::Value::Type::BINARY` (the JSON serializer can't handle it)
Instead:
* If the string is likely ASCII or UTF-8, use `NetLogStringValue()`.
* If the string is arbitrary data, use `NetLogBinaryValue()`.
* If the string is guaranteed to be valid UTF-8, you can use
`base::Value::Type::STRING`
Also consider the maximum size of any string parameters:
* If the string could be large, truncate or omit it when using the default
capture mode. Large strings should be relegated to the `kEverything`
capture mode.
### 64-bit integers
NetLog parameters are specified as a JSON serializable `base::Value` which does
not support 64-bit integers.
Be careful when using `base::Value::Dict::Set()` as it will truncate 64-bit
values to 32-bits.
Instead use `NetLogNumberValue()`.
### Backwards compatibility
There is no backwards compatibility requirement for NetLog events and their
parameters, so you are free to change their structure/value as needed.
That said, changing core events may have consequences for external consumers of
NetLogs, which rely on the structure and parameters to events for pretty
printing and log analysis.
The [NetLog viewer](https://netlog-viewer.appspot.com/) for instance pretty
prints certain parameters based on their names, and the event name that added
them.
### Example 1
Add an `PROXY_RESOLUTION_SERVICE` event without any parameters, at all capture
modes.
```
net_log.BeginEvent(NetLogEventType::PROXY_RESOLUTION_SERVICE);
```
Analysis:
* Privacy: Logging the event at all capture modes only reveals timing
information.
* Performance: When not logging, has the overhead of an unconditional function
call (`BeginEvent`), and then a branch (test on `IsCapturing()`).
* Size: Minimal data added to NetLog - just one parameterless event per URL
request.
### Example 2
Add a `FTP_COMMAND_SENT` event, at all capture modes, along with parameters
that describe the FTP command.
```
if (net_log.IsCapturing()) {
std::string command = BuildCommandForLog();
net_log.AddEventWithStringParams(NetLogEventType::FTP_COMMAND_SENT,
"command", command);
}
```
Analysis:
* Privacy: Low risk given FTP traffic is unencrypted. `BuildCommandForString()`
should additionally best-effort strip any identity information, as this is
being logged at all capture modes.
* Performance: Costs one branch when not capturing. The call to
`BuildCommandForString()` is only executed when capturing.
* Size: Cost is proportional to the average FTP command length and frequency of
FTP, both of which are low. `BuildCommandForLog()` needn't strictly bound the
string length. If a huge FTP command makes it to a NetLog, there is a good
chance that is the problem being debugged.
### Example 3
Add a `SSL_CERTIFICATES_RECEIVED` event, along with the full certificate chain,
at all capture modes.
```
net_log.AddEvent(NetLogEventType::SSL_CERTIFICATES_RECEIVED, [&] {
base::Value::Dict dict;
base::Value::List certs;
std::vector<std::string> encoded_chain;
server_cert_->GetPEMEncodedChain(&encoded_chain);
for (auto& pem : encoded_chain)
certs.Append(std::move(pem));
dict.Set("certificates", std::move(certs));
return base::Value(std::move(dict));
});
```
Analysis:
* Privacy: Low risk as server certificates are generally public data.
* Performance: Costs one branch when logging is off (hidden by template
expansion). The code in the lambda which builds the `base::Value` parameters is only
executed when capturing.
* Size: On average 8K worth of data per request (average of 2K/certificate,
chain length of 3, and the overhead of PEM-encoding). This is heavy-weight
for inclusion at `kDefault` capture mode, however justified based on how
useful the data is.
### Example 4
Add a `COOKIE_STORE_COOKIE_ADDED` event at all capture modes. Moreover, if the
capture mode is `kIncludeSensitive` or `kEverything`, also logs the cookie's
name and value.
```
net_log.AddEvent(NetLogEventType::COOKIE_STORE_COOKIE_ADDED,
[&](NetLogCaptureMode capture_mode) {
if (!NetLogCaptureIncludesSensitive(capture_mode))
return base::Value();
base::Value::Dict dict;
dict.Set("name", cookie->Name());
dict.Set("value", cookie->Value());
return base::Value(std::move(dict));
});
```
Analysis:
* Privacy: The cookie name and value are not included at the `kDefault` capture
mode, so only cookie counts and timing information is revealed.
* Performance: Costs one branch when logging is off (hidden by template
expansion). The code in the lambda which builds the `base::Value` parameters is only
executed when capturing.
* Size: For default captured logs, has a file size cost proportional to the
number of cookies added. This is borderline justifiable. It would be better
in this case to simply omit the event all together at `kDefault` than to log
a parameterless event, as the parameterless event is not broadly useful.

103
src/net/docs/pools.dot Normal file
View File

@@ -0,0 +1,103 @@
digraph SocketPools {
subgraph cluster_legend {
label="Legend";
## The following legend is an attempt to match UML notation,
## except for template_class and Factory->object, which are
## invented for this diagram.
BaseClass;
SubClass [label="Derived Class"];
Whole;
Part;
A;
B;
Interface [label="Interface / ABC", style=dashed];
template_class [shape=diamond]; # Link will name parameter(s)
SubClass -> BaseClass [arrowhead="empty"];
SubClass -> Interface [arrowhead="empty", style=dashed];
Part -> Whole [arrowhead="diamond", label="ownership"];
Part -> Whole [arrowhead="odiamond", label="pointer"];
RefCountedPart -> Whole [arrowhead="diamond", color=red,
label="partial\nownership"];
A -> B [arrowhead="none", headlabel="?..?", taillabel="?..?",
label="association"];
// Often a "subgraph { rank=same; .. }" is used to wrap the
// below to make the generative relationship distinctive
// from the other class relationships.
Factory -> object [arrowhead=veevee];
};
ClientSocketPoolBase [shape=diamond];
ClientSocketPoolBaseHelper;
ClientSocketPoolBaseHelper_ConnectJobFactory
[style=dotted, label="ClientSocketPoolBaseHelper::\nConnectJobFactory"];
ClientSocketPoolBase_ConnectJobFactory
[style=dotted, shape=diamond,
label="ClientSocketPoolBase::\nConnectJobFactory"];
ClientSocketPoolBase_ConnectJobFactoryAdaptor
[shape=diamond,
label="ClientSocketPoolBase::\nConnectJobFactoryAdaptor"];
HigherLayeredPool [style=dotted];
LowerLayeredPool [style=dotted];
ClientSocketPool [style=dotted];
ConnectJob [style=dashed];
ConnectJob_Delegate [style=dotted, label="ConnectJob::Delegate"];
ClientSocketFactory [style=dotted];
DefaultClientSocketFactory;
TCPClientSocket;
TransportClientSocket [style=dotted]
StreamSocket [style=dotted]
Socket;
TransportSocketParams;
TransportConnectJobHelper;
TransportConnectJobFactory;
TransportConnectJob;
TransportClientSocketPool -> ClientSocketPool [arrowhead=empty];
ClientSocketPool -> LowerLayeredPool [arrowhead=empty];
ClientSocketPoolBaseHelper -> ConnectJob_Delegate [arrowhead=empty];
TransportConnectJobFactory -> ClientSocketPoolBase_ConnectJobFactory
[arrowhead=empty, label="TransportSocketParams"];
ClientSocketPoolBase_ConnectJobFactoryAdaptor ->
ClientSocketPoolBaseHelper_ConnectJobFactory
[arrowhead=empty, arrowtail=none];
TransportConnectJob -> ConnectJob [arrowhead=empty];
DefaultClientSocketFactory -> ClientSocketFactory [arrowhead=empty];
StreamSocket -> Socket [arrowhead=empty]
TCPClientSocket -> TransportClientSocket [arrowhead=empty]
TransportClientSocket -> StreamSocket [arrowhead=empty]
ClientSocketPoolBaseHelper -> ClientSocketPoolBase [arrowhead=diamond];
ClientSocketPoolBase -> TransportClientSocketPool
[arrowhead=diamond, label="TransportSocketParams"];
ClientSocketPoolBase_ConnectJobFactory ->
ClientSocketPoolBase_ConnectJobFactoryAdaptor [arrowhead=diamond];
ClientSocketPoolBaseHelper_ConnectJobFactory ->
ClientSocketPoolBaseHelper [arrowhead=diamond];
TransportConnectJobHelper -> TransportConnectJob [arrowhead=diamond];
TransportSocketParams -> TransportConnectJobHelper
[arrowhead=diamond, color=red];
ConnectJob -> ConnectJob_Delegate
[dir=back, arrowhead=none, arrowtail=odiamond];
HigherLayeredPool -> ClientSocketPoolBaseHelper
[arrowhead=odiamond, taillabel="*"];
LowerLayeredPool -> ClientSocketPoolBaseHelper
[arrowhead=odiamond, taillabel="*"];
ClientSocketFactory -> ClientSocketPoolBaseHelper [arrowhead=odiamond];
subgraph {
rank=same;
ClientSocketPoolBaseHelper_ConnectJobFactory -> ConnectJob
[arrowhead=veevee];
}
ClientSocketPoolBase_ConnectJobFactory -> ConnectJob [arrowhead=veevee];
ClientSocketFactory -> TCPClientSocket [arrowhead=veevee]
}

BIN
src/net/docs/pools.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 210 KiB

1068
src/net/docs/proxy.md Normal file
View File

File diff suppressed because it is too large Load Diff

127
src/net/docs/supported-projects.md Normal file
View File

@@ -0,0 +1,127 @@
# Supported Projects
The `//net` stack is used on a variety of platforms and in a variety of open
source projects. These different platforms and projects have differing
degrees of support from `//net` OWNERS as well as differing requirements for
designs and technical requirements.
Note that this is a rough high-level overview of the major projects; as more
of `//net` is broken into consumable services/components as part of the Mojo
servicificaiton efforts, it's likely that there will be a larger number of
'variants' of these projects with different constraints and features.
## Google Chrome Browser
The Google Chrome browser, which lives in `//chrome`, is the most important
`//net` consumer and shapes many of the core design decisions. In general,
features that are not intended with or not compatible with the needs of
the Google Chrome browser will have a very high bar for acceptance in `//net`.
The feature matrix
* **Supported Platforms**: Windows, macOS, Linux, Chromium OS, iOS, Android
* **Release Frequency**: ~6 weeks between releases
* **Automatic Updates**: Yes
* **Command-line Flags**:
* __Yes__: Windows, macOS, Linux, Chromium OS (Dev image), Android (rooted)
* __No__: Chromium OS (Release image), iOS, Android (Release)
* **Field Trials (Finch)**: Yes
* **Enterprise Policy**: Yes
* **User Metrics (UMA)**: Yes
* **Component Updater**: Yes
## Chromium Browser
The Chromium browser refers to the practice of certain Linux distributions to
bundle the open-source components of Chrome Browser in `//chrome`, branded
as Chromium. This version is not distributed by Google, but by individual
Linux distributions (primarily). Other distributions based on building Chromium
for other platforms exist, although do not see as wide of usage.
* **Supported Platforms**: Windows, macOS, Linux, Chromium OS, iOS, Android
* **Release Frequency**: Varies by distributor; some Linux distributions
treat versions of Chromium as "Long Term Stable" and support a single
version for a longer time than the Chromium projects do, others track
the Google Chrome release frequency.
* **Automatic Updates**: Varies by distributor
* **Command-line Flags**:
* __Yes__: Windows, macOS, Linux, Chromium OS (dev image), Android (rooted)
* __No__: Chromium OS (Release image), iOS, Android (Release)
* **Field Trials (Finch)**: No
* **Enterprise Policy**: Yes
* **User Metrics (UMA)**: Varies by distributor
* **Component Updater**: Varies by distributor
## Android WebView
Distinct from the Chromium browser, the Android WebView is itself based on
the Chromium browser. On official Android devices running Android N or later,
WebView is automatically updated when Google Chrome is updated on the
device. For earlier devices, Android WebView is updated by the System WebView
component.
Android WebView may also be used on non-official Android devices, such as
those based on the Android Open Source Project but do not go through the
Android [Compatability Test Suite](https://source.android.com/compatibility/cts/).
Such releases have limited to no interaction with the Chromium projects, and
so their capabilities cannot be conclusively documented.
For official Android devices, WebView has the following capabilities.
* **Supported Platforms**: Android
* **Release Frequency**: ~6 weeks between releases
* **Automatic Updates**: Varies. Updates are made available on the Android
App Store, but users must explicitly choose to update. As such, the
rate of update varies much more than for the Chromium browser.
* **Command-line Flags**: No for production devices, [yes for userdebug
devices](https://chromium.googlesource.com/chromium/src/+/HEAD/android_webview/docs/commandline-flags.md)
* **Field Trials (Finch)**: Yes, [with
caveats](https://g3doc.corp.google.com/analysis/uma/g3doc/finch/platforms.md?cl=head)
* **Enterprise Policy**: Yes, with caveats (TODO(rsleevi): document caveats)
* **User Metrics (UMA)**: Yes, [with caveats](http://go/clank-webview/uma)
* **Component Updater**: No
## `//content` Embedders
In addition to Chromium, there are a number of other of embedders of
`//content`, such as projects like [Chromium Embedded Framework](https://bitbucket.org/chromiumembedded/cef),
[Electron](http://electron.atom.io/) or Fuchsia's [WebEngine](https://chromium.googlesource.com/chromium/src/+/HEAD/fuchsia_web/webengine/).
While `//net` does not directly support these consumers, it does support the
`//content` embedding API that these projects use. Note that this excludes the
[content_shell](../../content/shell) test framework.
* **Supported Platforms**: Windows, macOS, Linux, Chromium OS, iOS, Android,
Fuchsia
* **Release Frequency**: Varies by consumer; Officially ~6 weeks
* **Command-line Flags**: Varies by consumer
* **Field Trials (Finch)**: No
* **Enterprise Policy**: No
* **User Metrics (UMA)**: No
* **Component Updater**: No
## Cronet
[Cronet](../../components/cronet/README.md) is a version of the `//net`
network stack for use in mobile applications on iOS and Android. While
primarily targetting first-party Google applications, Cronet's status as an
open-source project, similar to the Chromium browser, means that it may
find itself embedded in a variety of other projects.
Unlike some of the other `//net` consumers, Cronet does not necessarily
implement "The Web Platform" (as the holistic set of user agent-focused
features), and instead is more akin to an HTTP(s) client networking library.
* **Supported Platforms**: iOS, Android
* **Release Frequency**: Varies. While "releases" are made following the
same frequency as Google Chrome, because it is treated similar to
a "third-party" library, different products and projects will ship
different versions of Cronet for differing periods of time.
* **Command-line Flags**: No
* **Field Trials (Finch)**: No
* **Enterprise Policy**: No
* **User Metrics (UMA)**:
* __Yes__: First-party (Google) projects, although they will be
reported in project-specific buckets (e.g. no overarching set of
metrics for all Cronet consumers).
* __No__: In general
* **Component Updater**: No

188
src/net/docs/url_request.dot Normal file
View File

@@ -0,0 +1,188 @@
digraph URLRequestRoot {
subgraph cluster_legend {
label="Legend";
## The following legend is an attempt to match UML notation,
## except for template_class and Factory->object, which are
## invented for this diagram.
BaseClass;
SubClass [label="Derived Class"];
Whole;
Part;
A;
B;
Interface [label="Interface / ABC", style=dashed];
template_class [shape=diamond]; # Link will name parameter(s)
SubClass -> BaseClass [arrowhead="empty"];
SubClass -> Interface [arrowhead="empty", style=dashed];
Part -> Whole [arrowhead="diamond", label="ownership"];
Part -> Whole [arrowhead="odiamond", label="pointer"];
A -> B [arrowhead="none", headlabel="?..?", taillabel="?..?",
label="association"];
// Often a "subgraph { rank=same; .. }" is used to wrap the
// below to make the generative relationship distinctive
// from the other class relationships.
Factory -> object [arrowhead=veevee];
};
## URLRequest, URLRequestJob, and subclasses
URLRequestContext;
URLRequest;
URLRequestJob [style=dashed];
URLRequestJob_Others [label="...other job types..."];
URLRequestHttpJob;
Filter;
{URLRequestHttpJob, URLRequestJob_Others} -> URLRequestJob
[arrowhead="empty"];
URLRequestJob -> URLRequest [arrowhead="diamond"];
Filter -> URLRequestJob [arrowhead="diamond"];
Filter -> Filter [arrowhead="diamond", taillabel="0..1"];
subgraph {
rank=same;
URLRequestContext -> URLRequest [arrowhead=veevee];
}
subgraph {
rank=same;
URLRequestHttpJob -> Filter [arrowhead=veevee];
}
## HttpTransaction, subclasses, and generative classes.
HttpTransactionFactory [style=dashed];
HttpCache;
HttpNetworkLayer;
HttpTransaction [style=dashed];
HttpCache_Transaction [label="HttpCache::Transaction"];
HttpNetworkTransaction;
{ HttpNetworkTransaction, HttpCache_Transaction } -> HttpTransaction
[style=dashed, arrowhead="empty"];
{ HttpNetworkLayer, HttpCache } -> HttpTransactionFactory
[arrowhead=empty, style=dashed];
HttpTransaction -> HttpCache_Transaction [arrowhead=diamond];
HttpTransaction -> URLRequestHttpJob [arrowhead="diamond"]
subgraph {
rank=same;
HttpCache -> HttpCache_Transaction [arrowhead=veevee];
}
subgraph {
rank=same;
HttpTransactionFactory -> HttpTransaction [arrowhead=veevee];
}
subgraph {
rank=same;
HttpNetworkLayer -> HttpNetworkTransaction [arrowhead=veevee];
}
## HttpStreamFactory and related.
HttpStreamFactory;
HttpStreamRequest;
HttpStream [style=dashed];
HttpStreamFactory_Job [label="HttpStreamFactory::Job"];
HttpStreamRequest_Delegate
[label="HttpStreamRequest::Delegate",style=dashed];
HttpBasicStream;
QuicHttpStream;
SpdyHttpStream;
HttpBasicState;
HttpNetworkTransaction -> HttpStreamRequest_Delegate
[style=dashed, arrowhead="empty"];
{ HttpBasicStream, QuicHttpStream, SpdyHttpStream } -> HttpStream
[style=dashed, arrowhead="empty"];
HttpStreamRequest -> HttpNetworkTransaction [arrowhead="diamond"];
HttpStream -> HttpNetworkTransaction [arrowhead="diamond"];
HttpBasicState -> HttpBasicStream [arrowhead=diamond];
HttpStreamFactory_Job -> HttpStreamRequest
[arrowhead="diamond",taillabel="1..*"];
HttpStreamRequest_Delegate -> HttpStreamRequest
[arrowhead=odiamond];
HttpStreamFactory_Job -> HttpStreamFactory_Job
[arrowhead=odiamond, label="blocking_job_\nwaiting_job_"];
subgraph {
rank=same;
HttpStreamFactory -> HttpStreamRequest [arrowhead=veevee];
}
subgraph {
rank=same;
HttpStreamRequest -> HttpStream [arrowhead=veevee];
}
## ClientSocketHandle and socket pools.
ClientSocketPool [style=dashed];
TransportClientSocketPool;
SSLClientSocketPool;
ClientSocketPool_Others [label="...others..."];
ClientSocketPoolBase [label="ClientSocketPoolBase", shape=diamond];
ClientSocketPoolBaseHelper;
ConnectJobFactory;
ConnectJob [style=dashed];
TransportConnectJob;
SSLConnectJob;
ConnectJob_Others [label="...other connect job types..."];
ConnectJob_Delegate [label="ConnectJob::Delegate",style=dashed];
StreamSocket [style=dashed];
TransportClientSocket [style=dashed];
TCPClientSocket;
StreamSocket_Others [label="...other socket types..."];
TransportConnectJobHelper;
SingleRequestHostResolver;
{ SSLClientSocketPool, TransportClientSocketPool,
ClientSocketPool_Others} -> ClientSocketPool
[style=dashed, arrowhead=empty];
ClientSocketPoolBaseHelper -> ConnectJob_Delegate
[arrowhead=empty, style=dashed];
StreamSocket -> Socket [arrowhead=empty, style=dashed];
TCPClientSocket -> TransportClientSocket
[arrowhead=empty, style=dashed];
TransportClientSocket -> StreamSocket
[arrowhead=empty, style=dashed];
StreamSocket_Others -> StreamSocket
[arrowhead=empty, style=dashed];
{SSLConnectJob, TransportConnectJob, ConnectJob_Others} -> ConnectJob
[style=dashed, arrowhead=empty];
ClientSocketHandle -> HttpStreamFactory_Job [arrowhead="diamond"];
ClientSocketHandle -> HttpBasicState [arrowhead="diamond"];
ClientSocketPoolBaseHelper -> ClientSocketPoolBase [arrowhead=diamond];
ClientSocketPoolBase -> TransportClientSocketPool
[arrowhead=diamond, label=TransportSocketParams];
ClientSocketPoolBase -> SSLClientSocketPool
[arrowhead=diamond, label=SSLSocketParams];
StreamSocket -> ClientSocketHandle [arrowhead=diamond];
ConnectJobFactory -> ClientSocketPoolBase [arrowhead=diamond];
StreamSocket -> ConnectJob [arrowhead=diamond];
SingleRequestHostResolver -> TransportConnectJobHelper
[arrowhead=diamond];
TransportConnectJobHelper -> TransportConnectJob [arrowhead=diamond];
ClientSocketPool -> ClientSocketHandle [arrowhead=odiamond];
ConnectJob_Delegate -> ConnectJob [arrowhead=odiamond];
subgraph {
rank=same;
ConnectJobFactory -> ConnectJob [arrowhead=veevee];
}
subgraph {
rank=same;
HttpStreamFactory_Job -> ClientSocketHandle [arrowhead=veevee];
}
subgraph {
rank=same;
TransportConnectJob -> StreamSocket [arrowhead=veevee];
}
}

BIN
src/net/docs/url_request.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 324 KiB