The Bits Must Flow -or- (Net)Working Through the Abstractions
A presentation at THAT Conference, WI 2023 in in Wisconsin Dells, WI, USA by fen aldrich
The Bits Must Flow netWorking through the abstractions @crayzeigh@hachyderm.io 1
@crayzeigh@hachyderm.io 2
@crayzeigh@hachyderm.io 3
@crayzeigh@hachyderm.io 4
Hi I’m fen (they/them) social: @crayzeigh@hachyderm.io slides: speaking.crayzeigh.com @crayzeigh@hachyderm.io 5
Developer Advocate: @crayzeigh@hachyderm.io 6
Internet Traffic Per Second 167,069 GB @crayzeigh@hachyderm.io 7
bits per second 1,336,544,000,000,000 @crayzeigh@hachyderm.io 8
What happens when you visit a website? @crayzeigh@hachyderm.io 9
@crayzeigh@hachyderm.io 10
OSI 7-Layer model Application Presentation Session Transport Network Data Link Physical @crayzeigh@hachyderm.io 11
OSI TCP/IP Application Application Presentation
Session
Transport Transport Network Internet Data Link Network Access Physical
@crayzeigh@hachyderm.io 12
Why all these layers anyway? @crayzeigh@hachyderm.io 13
Layer 2: Network Access Data Frames link digital to physical @crayzeigh@hachyderm.io 14
Layer 2: Switching Sending data to local devices @crayzeigh@hachyderm.io 15
Frame Header Preamble SFD Dest. MAC Src. MAC Type Data & Pad FCS 7 Bytes 1 Byte 6 Bytes 6 Bytes 2 Bytes 46–1500 Bytes 4 Bytes @crayzeigh@hachyderm.io 16
1 MAC Address 01:23:45 01:23:45:67:89:ab :67:89:ab 1. Media Access Control @crayzeigh@hachyderm.io 17
MAC Address – Identifies the (network) device – For devices on the local network @crayzeigh@hachyderm.io 18
Address Resolution Protocol @crayzeigh@hachyderm.io 19
ARP – Mapping IPs and MAC addresses @crayzeigh@hachyderm.io 20
ARP – Mapping IPs and MAC addresses – Necessary for your device to talk to your router @crayzeigh@hachyderm.io 21
ARP – Mapping IPs and MAC addresses – Necessary for your device to talk to your router – Finds neighbors through broadcast @crayzeigh@hachyderm.io 22
ARP – Mapping IPs and MAC addresses – Necessary for your device to talk to your router – Finds neighbors through broadcast – arp -a for your local table @crayzeigh@hachyderm.io 23
Virtual Local Area Networks @crayzeigh@hachyderm.io 24
VLANs – Used for limiting broadcast domains @crayzeigh@hachyderm.io 25
VLANs – Used for limiting broadcast domains – up to 4096 VLANs 1. 1 VXLAN addresses this limitation but that’s A Whole Other Thing @crayzeigh@hachyderm.io 26
VLANs – Used for limiting broadcast domains – up to 4096 VLANs 1 – Can be native (default traffic on a switch port) or tagged (logically divided in packet header) 1. VXLAN addresses this limitation but that’s A Whole Other Thing @crayzeigh@hachyderm.io 27
@crayzeigh@hachyderm.io 28
Layer 3: Internet Protocol Packets wrap your digital data and route it remotely @crayzeigh@hachyderm.io 29
Layer 3: Routing Directing data to remote destinations @crayzeigh@hachyderm.io 30
IPv4 Packet Header @crayzeigh@hachyderm.io 31
IP Address Classes and CIDR @crayzeigh@hachyderm.io 32
IP Classes & CIDR – Classless Inter-Domain Routing @crayzeigh@hachyderm.io 33
IP Classes & CIDR – Classless Inter-Domain Routing – Helps determine destination locality, i.e. routing @crayzeigh@hachyderm.io 34
IP Classes & CIDR – Classless Inter-Domain Routing – Helps determine destination locality, i.e. routing – CIDR replaced “class a/b/c” IP addressing to help address IP address availability @crayzeigh@hachyderm.io 35
CIDR Notation @crayzeigh@hachyderm.io 36
10.10.10.10/ 10.10.10.10/24 24 @crayzeigh@hachyderm.io 37
Host/ Host/Network Network bits @crayzeigh@hachyderm.io 38
10.10.10/ 10.10.10/24 24 IP Address: 10.10.10.10 Subnet Mask: 255.255.255.0 @crayzeigh@hachyderm.io 39
Converts to Binary IP: 00001010.00001010.00001010.00001010 SM: 11111111.11111111.11111111.00000000 In the Subnet Mask: 1 = Network 0 = Host @crayzeigh@hachyderm.io 40
Special IPs – Broadcast (ex. 10.10.10.255) – host bits are all 1’s – For sending data to all hosts in a network – Network (ex. 10.10.10.0) – host bits are all 0’s – only used for forwarding data between routers @crayzeigh@hachyderm.io 41
Putting it all together CIDR 10.10.10.10/24 Network 10.10.10.0/24 Broadcast IP 10.10.10.255 Available Host IPs 10.10.10.1 - 254 @crayzeigh@hachyderm.io 42
Bigger Networks CIDR 192.168.1.100/22 Network 192.168.0.0/22 Broadcast IP 192.168.3.255 Available Host IPs 192.168.0.1 - 192.168.3.254 @crayzeigh@hachyderm.io 43
Weird Ones @crayzeigh@hachyderm.io 44
Weird Ones Explained /30 – “Costs” 4 IPs, but only gives 2 host addresses – Broadcast & Network IPs still required – Used for legacy compatibility or you just really like holding IP addresses @crayzeigh@hachyderm.io 45
Weird Ones Explained /31 – Creates 2 adjacent host IPs – Only “costs” 2 IPs – Proposed in RFC3021 (in 2000) to combat dwindling IP availability @crayzeigh@hachyderm.io 46
Weird Ones Explained /32 – Single IP Address – useful generally for isolating public internet traffic @crayzeigh@hachyderm.io 47
IPv6 @crayzeigh@hachyderm.io 48
IPv6 Packet Header @crayzeigh@hachyderm.io 49
IPv6 – 128 bits long (vs 32 bits for v4) @crayzeigh@hachyderm.io 50
IPv6 – 128 bits long (vs 32 bits for v4) 32 9 2 ≈ 4.3x10 @crayzeigh@hachyderm.io 51
IPv6 – 128 bits long (vs 32 bits for v4) 32 9 2 ≈ 4.3x10 ≈ 4.3 billion @crayzeigh@hachyderm.io 52
IPv6 – 128 bits long (vs 32 bits for v4) 32 9 2 ≈ 4.3x10 ≈ 4.3 billion 128 28 2 ≈ 3.4x10 @crayzeigh@hachyderm.io 53
IPv6 – 128 bits long (vs 32 bits for v4) 32 9 2 ≈ 4.3x10 ≈ 4.3 billion 128 28 2 ≈ 3.4x10 ≈ 27 orders of magnitude larger @crayzeigh@hachyderm.io 54
IPv6 – 128 bits long (vs 32 bits for v4) – introduced to handle IP shortage – written as 8 groups of 4 bytes in hex @crayzeigh@hachyderm.io 55
Abbreviations fddd:f00d:b33f:0000:0000:0000:0000:0001 @crayzeigh@hachyderm.io 56
Abbreviations fddd:f00d:b33f:0:0:0:0:1 @crayzeigh@hachyderm.io 57
Abbreviations fddd:f00d:b33f::1 @crayzeigh@hachyderm.io 58
Abbreviations 00ff:0000:0000:0000:0001:0000:0000:0001 @crayzeigh@hachyderm.io 59
Which one is correct? ff:0:0:0:1:0:0:1 1. ff::1:0:0:1 2. ff::1::1 3. ff:0:0:0:1::1 @crayzeigh@hachyderm.io 60
Which one is correct? ff:0:0:0:1:0:0:1 1. ff::1:0:0:1 2. ff::1::1 (ambiguous) 1 3. ff:0:0:0:1::1 (longest 0’s must be shortened ) 1. RFC 5952 @crayzeigh@hachyderm.io 61
Subnetting @crayzeigh@hachyderm.io 62
Subnetting ifconfig | grep inet6 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet6 fe80::1470:43d6:1243:7a20%en0 prefixlen 64 secured scopeid 0x4 inet6 2601:19e:8380:20dc:4ea:6990:6d8e:68a3 prefixlen 64 autoconf secured inet6 2601:19e:8380:20dc:5ce:b0e:70ad:988 prefixlen 64 deprecated autoconf temporary inet6 2601:19e:8380:20dc::f42a prefixlen 64 dynamic inet6 2601:19e:8380:20dc:141b:7764:190d:9240 prefixlen 64 deprecated autoconf temporary inet6 2601:19e:8380:20dc:d477:fa12:2a44:7983 prefixlen 64 autoconf temporary … @crayzeigh@hachyderm.io 63
Subnetting address obtained ::1 loopback (special) fe80::1470:43d6:1243:7a20 self-assigned (local) 2601:19e:8380:20dc:4ea:6990:6d8e:68a3 self-assigned (global) 2601:19e:8380:20dc::f42a DHCPv6 (global) @crayzeigh@hachyderm.io 64
Routing Sending data to remote addresses @crayzeigh@hachyderm.io 65
Source: 10.10.10.10/24 Destination: 10.10.10.100 @crayzeigh@hachyderm.io 66
Source IP: 10.10.10 10.10.10.10 .10 Network: 255.255.255 255.255.255.0 .0 Destination: 10.10.10 10.10.10.100 .100 @crayzeigh@hachyderm.io 67
Source: 10.10.10.10/24 Destination: 10.10.10.100
- Checks network space to see address is local 2. Sends local ARP broadcast to find MAC of destination 3. wraps packet in frame with newly discovered MAC 4. sends data frame through switch to destination “directly” @crayzeigh@hachyderm.io 68
Source: 10.10.10.10/24 Destination: 1.1.1.1 @crayzeigh@hachyderm.io 69
Source IP: 10.10.10 10.10.10.10 .10 Network: 255.255.255 255.255.255.0 .0 Destination: 1.1.1 1.1.1.1 .1 @crayzeigh@hachyderm.io 70
Source: 10.10.10.10/24 Destination: 1.1.1.1
- Checks network space and see’s address is remote 2. Wraps packet in frame with router’s mac address and desired destination IP, forwards to router 3. Forwards frame to the router, router re-wraps packet with a frame pointing to the next router in line 4. And so on until the local router for 1.1.1.1 receives the packet and wraps in the final destination frame @crayzeigh@hachyderm.io 71
How does the router know where the next hop in line is? @crayzeigh@hachyderm.io 72
Routing Tables 3 Types of Routes: – Connected: The networks connected to your interfaces – Static: Manually set, e.g. Default Route/Gateway – Learned: Learned from advertising peers, e.g. via BGP @crayzeigh@hachyderm.io 73
BGP @crayzeigh@hachyderm.io 74
BGP @crayzeigh@hachyderm.io 75
BGP @crayzeigh@hachyderm.io 76
Anycast @crayzeigh@hachyderm.io 77
Anycast Benefits @crayzeigh@hachyderm.io 78
Anycast Benefits @crayzeigh@hachyderm.io 79
What Happens when you visit a website? @crayzeigh@hachyderm.io 80
@crayzeigh@hachyderm.io 81
Try it! use discount code THATCONF23 for $800 credit http://eqix.co/metal /assets/Simple_Footer_center.png @crayzeigh@hachyderm.io 82
Getting Started check out our youtube channel for tips and ideas http://eqix.co/start-metal @crayzeigh@hachyderm.io 83
@crayzeigh@hachyderm.io 84
thanks! social: @crayzeigh@hachyderm.io slides: speaking.crayzeigh.com @crayzeigh@hachyderm.io 85
organizer: @crayzeigh@hachyderm.io 86
host: @crayzeigh@hachyderm.io 87
@crayzeigh@hachyderm.io 88
As with all modern computing, the network stack has gotten increasingly abstracted away as we move to cloud services and cloud native infrastructure, but underneath it all, we’re still trying to accomplish the same things as always, get data from one system to another as quickly and efficiently as possible without interruption or eavesdropping. Starting with the classic interview question, “What happens when you try to access a website from your computer?” this talk reviews the modern complexity of the internet and reminds us about how some functional routing, switching and firewall knowledge can help us untangle the modern messes generated by layers of abstraction. We’ll start with some things like, “what about before I even get an IP address” and end up talking about BGP, the routing protocol that runs the internet. You’ll leave with a better understanding of what’s actually happening after you apply your chosen network settings to your cluster or hyperscaler, and a better framework for understanding how your cloud native applications and services are communicating across the web.
