Wednesday, May 22, 2019

xz man page

I either nominate xz for the worst man page ever or I gladly make a fool of my self for not understanding man pages.

There is no help on the man page for how to specify an input file with a separate output file name.

The man page goes on and on and on about different compression ratios and how to specify a custom compression profile.

But no mention of "-k" or "-c" or how to use them.

This is the sort of failure that our community needs to acknowledge.  This sort of error creates huge frustration for millions of people who mostly know what they are doing, and can completly stop new people from entering the community.

How do I fix this?  Can I submit a patch somehow to update the man page to make it more usefull?  -  {sigh}

EDIT: Ok, I was wrong.  "-k" and "-c" are mentioned.  I still find that reading this man page provides no understanding of how to use these options.  

This is how you compress a file and keep the origional:
xz -c original.file > compressed.file.xz
Hindsight is great eh?  Makes sense now.  -  But the point of a manual is to help the reader gain understanding.  man pages almost always fail at this.  They are good references to things which are already known, but terrible tools to learn from.

Monday, May 20, 2019

Bridge to the Shop

I tried out a 64bit Gentoo image for Raspberry Pi...    This is very Gentoo.  The Rapberry Pi 3B+ has a 64bit processor, but 32 interface to, like, everything.  Basicially no one runs a 64bit OS on the Raspberry Pi, but you could, and if you use Gentoo and you can do something, well why not?!

It ran very fast for a couple of minutes, then it crashed.

I went back to Raspbian, and ran into a problem.  One of my Raspbian images was running an updated kernel, that did not have anything in /lib/modules that matched.  So I was unable to update the WiFi drivers.  I found an article with the correct command to reinstall the Raspbian Kernel:

sudo apt-get install --reinstall raspberrypi-bootloader raspberrypi-kernel

So play time over and annoying problem accomplished I moved on to thinking about how to actually implement the bridge.  I think I'm going to setup two RaspberryPi's to make this bridge.  That will allow it to be more portable, and might make creating a educational guide for others to learn from easier.  (The alternative being to use the RPi access point that I already have as an end point.)

L2TPv3 (Layer 2 Tunneling Protocol Version 3) seems to be the key piece of technology that I need to make this work. :-)  I found that has a very succieient explination of L2TPv3.  While Wikipedia's L2TPv3 entry has some interesting history.

Sunday, May 19, 2019

Mesh Networking

I've started down a long road.   I didn't know how long a road this was until I got on it and started walking.  This road is difficult to travel because it's not clear where the road is actually going and all the signposts have fallen over and there are angry bushes with thorns growing atop them.  :-)

Garage Network Connection - Goal #1

Make a WiFi bridge between my home network and the network in the shop/garage that just works like two network switches plugged into each other.

Status: Mostly failure.  I've learned a lot but don't have anything functional yet.  It's been a while since I worked in earnest on this goal.  I feel like I have a real chance of getting there.

Problems discovered and discussion of solutions:

Layer 3 vs Layer 2

 WiFi is a layer 3 technology, but networks need layer 2 connectivity if your going to have multiple devices on both sides of a connection.  Solution: make a WiFi connection between two devices, and route traffic between the two networks through that connection.  This should work, but I ran into a routing hardware problem

Routing Hardware Problem

My ~13+ year old WiFi router has a configuration page for routing subnets.  But it's a sham, total baloney.  It's not capable of routing subnets.  Solution: Ubiquity Edge Router X - Fixed. Done! Victory!  Except that might not be good enough because of "IOT" crap software and hardware that is out of my control.

The IOT problem.

We use a Chromecast to watch TV, play music, etc.  If you want to cast some content from your cell phone or tablet the way to connect is to use the google home app.  To add a chromecast you click the pretty little blue button.  You have no way of telling the home app to look for a chromecast on a different subnet.  Basicially dumb internet devices really cause headaches when your home network doesn't look "normal".  Could the solution be B.A.T.M.A.N.?  I don't know but that led me straight into my next goal before I had finished this one.  Which is not great.  Maybe I can get back to the original goal now, and then deal with the IOT problem later.

Seamless WiFi and/or Mesh Network - Goal #2

This could fix the IOT problem.  It was always a goal to install multiple WiFi AP's around the house, the shop, the future gazebo, the firepit, the front yard, etc.  It's frustrating trying to listen to internet radio under my earmuffs while mowing the yard because I'm constantly loosing and regaining wifi as I travel back and forth with the mower.  Also when friends come over I want to be able to help them connect to WiFi ONCE, and not frustrate them or me with having to connect over and over to different WiFi networks as we move about.  So I need seamless WiFi.  If I can also get a WiFi mesh going that would limit the number of buried ethernet cables that I have to trench.  Also learning about Mesh networks and the technologies that are being developed for community use would be awesome!  Maybe I can help create a local Mesh Network!

The network switch problem

 Turns out that the SBC's I love so much (single board computers - mostly raspberry Pi's) generally don't have network switch hardware, and their WiFi devices are generally connected via USB.  This is apparently awful for creating mesh network nodes.  This limits a nodes usefulness to connecting to at most 5-7 other neighbors and clients.  The WiFi routers that we buy at the supermarket not only come with questionable proprietary software, but also hardware network switches connected (more or less) directly to WiFi chipsets.  There are two solutions:

Liberate commercial WiFi Router devices, or buy specialty network hardware - then install and configure them with libre/open source software.

Using the proprietary hardware is unacceptable to me at this point.  13 years ago when I bought my DLink DIR-655 spying on customers/consumers was just on the horizon of becoming a thing.  Today we have to fight for privacy against every company and with every chip that we bring into our houses.  I do have my guilty pleasures, there are some privacy battles I choose to loose for convienience or for fun.  I use Goolge almost everything, cause it's so useful; and I have a pair of Vinci Headphones my Dad gave me.  They supposedly listen and learn what I like and use neural network algorithms to choose what music to play... or something... /rant

So, for small number of users and not mesh use the RaspberryPi's should be fine.  For community level mesh networking we will need better hardware, like the Espressobin.  I bought one for testing, I'm sure I'll blog about it eventually.

The seamless problem

I haven't even gotten into this problem yet.  I know it can be done, but the whole mesh thing is a rabbit hole that has been pulling me down in.  I think that a good mesh solution will also solve the multiple WiFi AP transparent hopping problem so I've been looking into it from that perspective.  But because of how difficult that might be, I might back off and try to solve this first.  We'll see.

The Mesh problem

Manufactures have been solving this, or half, solving this with proprietary solutions for a while now.  Also it's been so slow in coming that there is actually a conference in Paris called Batlemesh where different mesh solutions compete.  I've just started to learn about what the diferent problems there are and what each technology is meant to do and how they fit together. 

There might be hope on the horizon in 802.11s.  Unfortunately there are VERY few WiFi chipsets with software drivers/kernel-modules combinations that support 802.11s.  There are a new 'N' class devices out there, but as I said in a previous post I'd really like to upgrade to faster 'AC' wifi.

Tomesh has a very easy to install solution for RapberryPi's - but they do all their user education face to face in Toronto.  They have a great install script, but it asks questions about all this stuff, and there are no links to documentation or ANY explanation as to what each of the modules is for or what it does. - So that's another long road...

LibreMesh looks amazing, but it is lazer focused on liberated WiFi router hardware, which is great, but I'm stuck with Raspberry Pi's right now and the RaspberryPi image they have is broken.

Then there is stuff like blogs and almost how-to's on how to setup WiFi mesh nodes.  This might be useful to read, and maybe I can glean some context and specific tid-bits from these type of articles.  But I'm done wasting my time following How-To's that don't educate more than they instruct.

A lesser solution might be Ad-Hoc networks...  I'm not really sure.  The pi-adoc-mqtt-cluster wiki looks promising.

That's my whole day and a half gone.  Writing this all down is definitely going to help me in the future, who knows maybe it helps someone else too. :-)