Pages

2016-05-07

Installing and configuring NAS

Motive: I already had some 7 computers and 2 e-book readers in household. I am working on 2-3-4 different machines... - file syncing and backuping started to bother me ... if we put it mildly. Problem was somehow aggravated by unholy mixup of systems: xp, 7, wheezy, antix, slackware etc.
Idea: Obvious - build a NAS for centralized sync and backup.
Easier said than done.

Hardware, very nonpro, as I didn't want to spend hugely for SOHO  solution...
MB: H110I-plus D3
CPU: Pentium
Memory: 16GB non-ecc,
OS: on 16GB USB stick,
HDD: WD red NASware, 4x2TB,
Ethernet: onboard realtek 8111H (and additional Intel NIC for some lagging),
Case: Bitfenix' Phenom (real nice, heavy metal, no bending, easy hdd-selves, good layout).
Build went quite smoothly - with one hickup: (small) cpu-cooler I bought was still a bit wide for cramped ITX. Intel stock cooler it was, then.

OP-system
Note: I am total noob in both NAS and BSD - the following AAR is therefore kinda simplistic, erratic and occasionally silly.
There were some reasons for not picking Linux - like I wanted to try BSD, and I didn't find satisfactory Linux-solution (whatever that means).
I started with current Nas4free. It installed nicely from usb to usb, booted (not very fast) ... and didn't have usable network driver. And I am talking - maybe - about a month of waiting, as freebsd 10.3 (with proper driver) is already released, but nas4free base is still 10.2 ... But I didn't feel like waiting and/or compiling patched driver in jail etc:
Freenas 9.10 (freebsd kernel 10.3) got downloaded, dd-d, installed, and it booted (a bit faster) with realtek support.
Install was uneventful - and went very much according to manual.
Then I shut down, connected HDDs to motherboard and rebooted.
And discovered the one problem with this install - new intel H110 chipset doesn't want to boot USB-system. Fortunately, the motherboards' bios has 'force boot from usb' option. So, F2 every time, and monitor+keyboard have to be close... Irritating, but survivable. Solution would be to use small SSD for system - but then we loose one SATA port (and PCIe sdd-s are sillily expencive).

When adding such a - well, revolutionary - part to old, self-accumulated home network, it's also good time to check the old shit over. My oldest is an  AMD X3 from 8 years past. And not to forget switches... I did check - and replaced one 10/100 switch and added one 1000 NIC to old AMD-box. No need to leave bottlenecks in new shiny installation.

As hardware building was easy and installing a tiny bit less so, then the following was quite frustrating and enormously time-google-wasting.

Configuration

# System
Now - it's possible to do quite a lot from webGUI, remotely. Still, there are some things which are better by console. So, one has to know at least some bsd commands and syntax (default csh is not unavoidable - bash is also installed).
WebGUI is big - so I am not going to blab of it. Read the same aforementioned manual. It's comprehensive. Not reciting manual goes also for text below.
I confed System first, then Storage (created volume and datasets), then user accounts. I planned to use both NFS and CIFS/Samba (afterthought - in mixed systems, maybe it's easier to stick with one - Samba). NFS - seems - wants your owner-user to have the same ID and GID as in client machine, which can be quite tiresome to achieve. If you have several differently dedicated datasets, then you probably also have to play around with groups - to give your users access to different places.
Also, NFS and some Windowses have relation-problems (win7pro doesn't recognize NFS natively, for example).
Next, Network, then Services (in my case: cifs, nfs, rsync and smart for starters) and then Sharing. The last one is absolutely essential - sharing opens datasets up for remote access. Among other conf options - here it's possible to close sharing(s) down for only LAN access, and/or for specific IPs.
This made me reorganize some of my boxes' IPs from dynamic to static - additional small hassle.

# CIFS/Samba
1. Additionally to (relatively simple) setting up CIFS service and share in NAS, samba(clients) need to be set up in client-machines. In Win7 it's as simple as connecting to server via explorers' network mapping. In Slackware there has to be - additionally to default-installed samba - valid /etc/samba/samba.conf. Daemon - /etc/rc.d/rc.samba does not need to be started, though.
I used Spacefms' plugin to mount samba shares. Doing it manually looks like this:
sudo smbclient //192.168.1.x/foldername -U username
The same can be achievad also with proper entry in /etc/fstab and then, if 'noauto' - mount /mnt/sambamountpoint.

# NFS
The same here, of course - client-machines have to be able to communicate with nfs-server. As mentioned, there is no satisfactory free solution for Win7Pro. In Slackware (in Antix 15.1 - nothing needs to be done, it works), start rpc daemon: sudo chmod 755 /etc/rc.d/rc.rpc && /etc/rc.d/rc.rpc start,
then, terminal way:
sudo mount -t nfs 192.168.1.x:/mnt/volume/dataset /mnt/mountpointname
fstab way looks something like that:
192.168.1.x:/mnt/volume/dataset /media/mountpointname nfs rw,noauto,noatime,user 0 0
So, afterwards it's only mount /media/mountpointname. Mount command for this can also be made in Spacefm as a, well, saved 'command'.
About confing: As I have couple of users for NFS, then I had to use 'mapall' in NAS' NFS-sharing options. Took me some time to figure that out. Also - I use nfs as passwordless 'guest' domain.

With both - smb and nfs - it's quite a lot of playing with user permissions - especially after in comes 'syncthing'.

# SYNCTHING
... comes with Freenas as an installable plugin-script. Script downloads stuff and installs it to jail. That ends the easy part.
For me, syncthing was the biggest time-waster to configure. It's wiki is not by far as comprehensive as Freenas one; it has bugs, it has quirks.
Still, after I got it running, I quite like it. I stress, though - be very careful, have a backup before experimenting! Syncing means that fucked-up (you, bug, ...) folder can fuck up all other synced folders... For important folders, do check versioning - if worst happens, you can get your shit back from trashcan - .stversions folder will be created when deletion occurs.

Syncthing creates user 'syncthing' (983:983) in jail. This complicates things with perms - some manual chowning needs to be done, myuser (same id-s as in dataset) needs to be created in jail; and syncthing should be member of this user group... Supposedly, it's easier to change 'syncthing' to have the same id/gid as main owner of dataset. This requires changing of conf: /var/db/syncthing/config.xml, otherways - console-based changes will be overwritten by GUI.
When creating datasets and connecting them to syncthing (can be quite time-consuming anyway)- avoid mistakes and renaming datasets, changes afterwards seems to drive syncthing totally bonkers.
In GUI, everything is reasonably clear: pick your folders, apply various options, sync. Yeah, right - but it took me 4 evenings to get all perms between win/linux/nas/syncthing correct. Syncthing forum, though, gives some nice advice about various obstacles.

And last - which should be the first - thing: Make plan/schema what and how to sync. I mean, really plan - as some changes are shit to do and, can fuck up good part of your already confed syncing.
This means: organizing location and naming of folders; which of synced folders is the master, how many boxes are going to access this dataset, how data really flows (do you really need to install one more syncthing, or maybe flow is one-directional - and rsync is enough) and so on ...

Moral: Syncthing is nice thing - if and when you get it smoothly running. To get it there might be enormous wrangling, though. I presume that some sysadmin experience before is certainly helpful...

And general moral: NAS is nice thing to have, and Freenas as its' op-system looks perfect to me.

2015-10-18

Lenovo S20-30 Touch + Crux 3.1

PART 3  | See Part 2 and Part 1

So, Crux is meant to be keeper. And this post will be long and occasionally detailed - as Crux' manual is not very long and help is only in form of mailing-list.

Made usb installer: dd if=crux-3.1.iso of=/dev/sdd bs=1M, booted and had auto-login as root.
Did whole fdisk /dev/sda, mkswap/swapon, mkfs -t ext4 /dev/sda2 (and sda3) again.
mount /dev/sda2 /mnt    #sda1=swap, sda2=/, sda3=all stuff
setup

Picked packages manually - left some things out. Installer finished with no errors and then it was time to chroot.
It's not particularly important in which order you do things when chrooted - and some of following can also be done after reboot.
setup-chroot  #script with all needed chroot-things
passwd  #create root password
adduser myuser  #can be done later but I did then
mkdir /media/back  #mount point for sda3
chown myuser:users /media/back
nano /etc/fstab  #has to be fixed, otherways - no boot.
nano /etc/rc.conf  #can be done later, but why wait:
FONT= default
KEYMAP=us   #all availables are in /usr/share/kbd/keymaps
TIMEZONE=Narnia/Mytown  # /usr/share/zoneinfo/
HOSTNAME=cruxlap
SYSLOG=sysklogd
SERVICES=(crond net)  #here go all started services, e.g: alsa gpm ntp sshd
localedef -i en_US -f UTF-8 en_US.UTF-8
cp /usr/share/zoneinfo/America/Yourplace /etc/localtime
hwclock --set --date="2012-04-19 16:45:05" --localtime
hwclock --hctosys

# I have habit to use localtime - because of windows dualboot in my desktop-machine. If Linux is single-system, then it's no problem using UTC.

Created udev/rules.d/70-persistent-net.rules with card mac number as eth0 - otherways udev changes card-name to something - and net is not going to load automatically on boot.

Kernel compilation.
Thought to be clever - and compiled newer kernel (3.18.22) than one provided by Crux. Well, it hanged when booting. Tried changing things / recompiling - but nothing worked. Copied confs to sda3 and reinstalled Crux (to be safe of leftovers). Copied confs back, and compiled original kernel.
Which booted alright. But - surely I was in some totally retarded mode - as I left Intels' video driver out. Recompile. Then I discovered that I left out also sound drivers. Recompile. And afterwards I discovered that I left out i2c - needed by coretemp... well, not essential - no recompile.
Anyway, compiling went exactly like it's written in Crux' page:
cd /usr/src/linux-3.12.24
make menuconfig
make all
make modules_install
cp arch/x86_64/boot/bzImage /boot/vmlinuz
cp System.map /boot


grub-install /dev/sda  &&  grub-mkconfig > /boot/grub/grub.cfg
I didn't like what grub created - so I made my own simple cfg file (can be copied from Crux' page, and then modified).
Copied all my desktop-Crux user-confs to laptop and changed ownership to my user.
exit && reboot

mount /dev/sda3 /media/back  #if not already in fstab
Before starting with updates I changed ports confs - made /media/back/ports and media/back/pac folders and edited /etc/pkgmk.conf and /etc/prt-get.conf accordingly. Also - every .rsync file in /etc/ports has to be changed too.
The point is - all sources+compilations grow very big, very soon. No need to stuff your root.
mv contrib.rsync.inactive contrib.rsync  #enable contrib, also in prt-get.conf
ports -u  #update ports list
prt-get sysup  #whole system upgrade - takes hours!

Only error I had was with xorg-server: missing libepoxy. Installed that and xorg compiled OK.

And then it was installing everything I normally do install. Troubles:

- gpm  # Only thing that worked was usb-mouse. Hacked /etc/rc.d/gpm:
/usr/sbin/gpm -m /dev/input/mice -t imps2
stop/start gpm and all touch-things started to function.
- openbox # cp /media/back/ports/openbox media/back/pac, then edited Pkgfile '--enable-imlib2 \' and also added imlib2 to depends line. The same goes for conky.
- xdotool  # Now there is package to be taken (1,5 years ago there wasn't)
- man needs pager - put into .bashrc: export pager='less'
- tint2 # has waken from long coma - and there is new version 0.12.2
Url in Pkgfile didn't work, though - the one in slackbuilds.org did, so I downloaded tar from there.  
- wmctrl  # url is slightly wrong - /dist has to be taken out.
- volumeicon 4.6  # last version with gtk2, needs patching.
- pale moon binary package to /opt, ln -s /usr/local/bin (I have a habit to put apps to local/bin... In Crux, this folder has to be created.)
- mplayer + smplayer  # Like previous time, there were severe problems. Basically , smplayer didn't play anything and whined about mplayer error. To cut it short - after almost 3 hours of aborted compilations I installed mpv, smplayer started to work with mplayer BUT not with mpv... &^%&#$$#^.
- compton  # Packages point to very old version. I dowloaded new git version but didn't succeed with make install (missing some dep). So I was impatient and ended up with just make and cp compton /usr/local/bin. And it works alright.
- mtpaint  # 3.40 doesn't compile - libpng-16 is too new for it. Git-version compiles ok.
- libunibreak  # is newer liblinebreak. There is no package in Crux - but it's easy to make. Fbreader needs it - and fbreaders' Pkgfile needs then fixing (liblinebreak out).
- libtorrent-rasterbar  # For qbittorrent. But - 1.0.5 from Pkgfile is a WRONG one. Qbittorrent needs 1.0.6 from git.
- qbittorrent  # Refused to make package (error - package is empty) - BUT at the same time - installed it.

And so on... Installing/upgrading Crux and adding your stuff takes 30 hours or probably more. But when all is done - it's real fast and lean. Exactly what fits with shity Celeron.

Lenovo S20-30 Touch with Funtoo and Alpine

PART 2  | See part 1

2. Funtoo, from 21. Sept stage 3 tarball.
Install went as per manual.
- Made systemrescue disk, also as per manual.
- Booted, used stage3-intel64-silvermont-pure64-funtoo-current-2015-09-21.tar.xz
Fixed fstab and some other stuff, rebooted successfully - but had no networking. Did:
rc-update add dhcpcd default && rc

Some things to write into make.conf (I am still talking of specific Lenovo netbook):
-CFLAGS="-march=native -O2 -pipe"
CXXFLAGS="-march=native -O2 -pipe"
MAKEOPTS="-j2"
-VIDEO_CARDS="intel"
-INPUT_DEVICES="evdev synaptics"


And then I started with defining USE flags ('emerge app-portage/ufed', an helper app)...
After spending some hours with flag-lists, and reading a bit more about vast system of emerge I came to conclusion that I do not want to build advanced spaceship and make 18 trips to Jupiter JUST for using a distro on my ships console.
I suppose it's bad influence of Slackwares' and Cruxs' minimalistic package-system approach... but really... is such a system of building life from premordial soup neccessary?
Not for me - format.


3. Alpine Linux 3.2.3 64bit.
Installation instructions in Alpines' site are ... a bit unclear... you almost think that to install it, you already HAVE to have Alpine installation, and no other way.
Fortunately, it can be done without:
Unetbootin made nice working usb-image (and dd didn't!). Booted it and:
login: root / 'enter'
(and the usual warning - MY root-part is sda2, YOURS might be different)

mount -t ext4 /dev/sda2 /mnt
setup-timezone
setup-alpine -q
setup-sshd  # if you are going to remote into it
setup-ntp  # if you want to use time from network


setup-disk -m sys /mnt

Now, why I said things about fdisk etc? Because I forgot to format properly, and ended up with borked installation. Syslinux didn't want to live in grub-infested MBR.
Fixed it with:
- dd if=/usr/share/syslinux/mbr.bin of=/dev/sda bs=440 count=1
Then some confing ... aaand ... boot succeeded.

adduser myuser;
edited /etc/group , added new user to users, disk, wheel, audio, video, cdrom, plugdev (the last one has to be made - spacefm needs it)
apk update && apk add sudo
enabled wheel in /etc/sudoers
apk add bash bash-doc bash completion  # if ash is not your favourite...
changed user-shell to bash in /etc/passwd

... read Alpines' helps, package lists and installed various things.

Then I did change main repos to 'edge' (means testing). I certainly do not advice to do that, they are testing for reason. But - I wanted my favs - spacefm, tint2 etc. Did:
apk upgrade
/etc/apk/repositories  # changed repo to edge
apk upgrade --update-cache --available
sync
reboot

... And there wasn't any net. Imbecilic udev had changed cards' names.
I made file /etc/udev/rules.d/80-net-setup-link.rules with mac number (which can be found with ifconfig), and for good measure, did:
ln -sf /dev/null /etc/udev/rules.d/80-net-name-list.rules
After that, it's easiest to reboot.
Once after that I lost network again - and discovered that it was because kernel time had reverted to UTC and was three hours in future...
Did 'date' and 'hwclock' to diagnose - first was wrong, the second was right, so:
hwclock --hctosys

Then I started to have some problems I didn't know how to fix.
And it all culminated with X starting without any input but usb-mouse (yes, yes - I did install evdev and synaptic, I did try to X -configure...).
Lot of curse-words, and ... yes, format it was.
Moral: Alpine  is recommended as lean and fast distro - provided that you stay with 'main' repo AND do not have touch-things. Also - can't have overly specific software needs...

Lenovo S20-30 Touch + Linux

Turning little Win-8.1-laptop to Linux-laptop.

Some relevant (drivers etc) technical data:
Resolution - 1366x768,
CPU - Celeron N2840 2,16 ghz 64bit Silvermont,
Graphics -  Intel HD,
Sound - Intel + realtek,
Ethernet - Realtek RTL8101E/RTL8102E,
Wifi - Qualcomm Atheros QCA9565/AR9565,
Touchpad - Elan.
Most of info came from lspci, lsusb, lscpu and inxi.

Original state.
I bought this little thing (11.6") in December 2014.
It had 'Windows 8.1 with Bing' OEM x86_64 installed (original dvd: IR3_CCONA_X64FREO_EN-US_DV9.iso), AND totally bloated with Lenovo crapware, including infamous Lenovo superfish-spyware (this link, or other thousands).
After struggling a bit with remove-all-shit, I tried - as a passtime - to install Slackware 14.1 for dual-boot. With partial success only - X refused to start (might have been problem of oldish kernel).
Fortunately, in process, I messed up EFI and Windows didn't start anymore - so I had an excuse to reinstall vanilla win8.1 (registration is automatic, in bios).
Tuned it, played around, and hated whole thing. You can't make bread out of shit, even with helpful addons. Also, whole system tended to be quite sluggish (not a surprise really, with Celeron N2840).
So, after a long while (it laying around mostly unused) I decided to wipe whole disk and install Linux.

The machine has efi+secure boot, but - BOTH can be disabled. And that makes installing Linux markedly easier.
My initial goals were something like:
- fast and lean distro; with little bloat (no fullblown desktop environment);
- reasonably stable;
- with all my favourite software available;
- without let-us-be-new-Windows like crap - from systemd to gnome shell.
Which made me pick Crux (crux.nu and do local search to find my three previous Crux-posts). But - because I also had a waiting-list of 'I should try this one', then I decided to try first those of the list (Antix 15, Funtoo, Alpine 3.2.3).

To action:

1. Antix 15, 64 killah p. 
A distro based on Debian 8 Jessie, but with systemd carefully removed. Full iso comes with four window managers which are swappable on fly, in X. It includes control center and various other scripts. Means - sounds interesting.

Antix' install is from the live session (I think I dd-d the usb-stick...).
username/passwd - demo/demo.
I deleted all Lenovo partitions, made swap, root and home partitions: traditional MBR, as I don't see why I should use GPT with one 500GB disk. And mind - after several installs and different bootloaders it's wise to use fdisk, gdisk, mkfs and such - for wiping disk really clean.
With little helpful trick - reading ability - the installation is a cakewalk, nothing need to be explained here.
Rebooted and logged to JWM.
Checked other WMs ... didn't like default confs and looks, as it always happens with me... So, here come short bits as 'How to tune debian distro, and also brake it':

... Synaptics, enabled more repos. Installed xapian-index (gives a search-window on toolbar);
... rebuilt JWMs menu and launchbar, reordered everything (Joe's site is very helpful as of howto);
... ~/.jwm/theme - seems to have a syntax error: not 'dejavu sans 10', but 'dejavu sans-10'. Otherways it doesn't react to any size change.

There were several annoyances from the beginning:
- Cursor jumping around when typing (very &^%&^#@$%!).
- medit and geany started occasionally new window instances - as opposed to keeping things in tabs...
- when changing window managers various things happened, including some apps closing, some forgetting tabs... But here I doubt very much if it's possible at all to smoothly integrate very different WMs and (different) default apps ...
- few usual app-related-bugs every distro tends to have: Here - clipit didn't start and xmahhjong crashed whole JWM.

So I started to remove Antix goodies, first other WMs and then all antix-scripts and antix-libs.
Re-tuned confs and shit... and, of course, ended up with one-or-two hidden, not-so-easily-findable leftover-quirks ...
My curiosity about Antix satisfied and not willing to spend more time with detective-work I formatted the whole thing.
Some general moral here: If not having unlimited time - do not full-rebuild prebuilt distro.
Ah, well... but it WAS interesting when it lasted. :)

As long you do not have touchscreen and -pad; and do not change 'desktops' on fly - Antix is recommended. It's nice, lean piece of work.
It also ended up as my 6-year old son desktop system. Who seems to be entirely OK with it.