FreeBSD upgrade pool ‘zroot’

Today I successfully upgraded my FreeBSD home nas server from 10.3 to 11.0. This is the final release of version 11.0, though the official announcement is expected to be made on September 28.

After the system upgrade, I need to also upgrade the 2 zpools (tank and zroot) so they can have new features. Upgrading tank was easy, all I needed to do was running this command:

# zpool upgrade tank
This system supports ZFS pool feature flags.

Enabled the following features on 'tank':
  sha512
  skein

For zroot, in addition to running the above command (by replacing the actually zpool name to zroot), I also need to update the boot code.

root@nas:~ # zpool upgrade zroot
This system supports ZFS pool feature flags.

Enabled the following features on 'zroot':
  sha512
  skein

If you boot from pool 'zroot', don't forget to update boot code.
Assuming you use GPT partitioning and da0 is your boot disk
the following command will do it:

        gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0

What’s the boot code? Good question. Not sure what it is, I’ll find out later by reading the FreeBSD document.

The boot disk in my server is not da0. How do we find out what it is?

root@nas:~ # gpart show
=>       34  125045357  ada4  GPT  (60G)
         34       1024     1  freebsd-boot  (512K)
       1058    4194304     2  freebsd-swap  (2.0G)
    4195362  120850029     3  freebsd-zfs  (58G)

In my case, it’s ada4, and the partition the boot sits on is ada4p1.
So, I can now proceed to update the boot code:

root@nas:~ # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada4
partcode written to ada4p1
bootcode written to ada4

Reboot the machine, and voilla it’s “still” working.

Perl IO::Socket::INET6::sockaddr_in6 redefined error

Yesterday I installed spamassassin package on a server running RHEL 6.8. Then this morning, I received an email alert with the following message:

Subroutine IO::Socket::INET6::sockaddr_in6 redefined at /usr/share/perl5/Exporter.pm line 67.
 at /usr/share/perl5/IO/Socket/INET6.pm line 21.
Subroutine IO::Socket::INET6::sockaddr_in6 redefined at /usr/share/perl5/Exporter.pm line 67.
 at /usr/share/perl5/IO/Socket/INET6.pm line 21.
Subroutine IO::Socket::INET6::sockaddr_in6 redefined at /usr/share/perl5/Exporter.pm line 67.
 at /usr/share/perl5/IO/Socket/INET6.pm line 21.
Subroutine IO::Socket::INET6::sockaddr_in6 redefined at /usr/share/perl5/Exporter.pm line 67.
 at /usr/share/perl5/IO/Socket/INET6.pm line 21.

This seems to be a bug exists in an old version of IO::Socket::INET6. Upgrading this package should fix the problem:

# cpan IO::Socket::INET6

Ref: http://stackoverflow.com/questions/23909848/perl-iosocketinet6sockaddr-in6-redefined-error

Warning: Use of “localtime” without parentheses is ambiguous

While looking at the log produced by Request Tracker (RT) 4.2.12, I noticed there was a warning message about using “localtime” without parentheses.

Here’s the actual error message:

[warning]: Warning: Use of "localtime" without parentheses is ambiguous 
at /opt/rt4/sbin/../lib/RT/Interface/Email.pm line 526.

The solution which I found from here, fixed the problem:

On line 526 of file /opt/rt4/sbin/../lib/RT/Interface/Email.pm, replace:

localtime with localtime().

Thanks to the chap who posted the solution.

Losing ZFS storage for Docker

I use ZFS as a storage driver for docker engine running on my machine. Today after my machine rebooted from a crash, yes Linux system crashes too, I notice that all my docker images and containers disappeared.

~ ❯❯❯ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
~ ❯❯❯ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

First thing came to my mind was “did I accidentally destroy docker zfs data set last night?”

# zfs list
NAME                                                                                USED  AVAIL  REFER  MOUNTPOINT
tank                                                                               1.52T   239G   120K  /tank
tank/docker                                                                         999M   239G  73.9M  /var/lib/docker

It’s still there. At that point I suspected that Docker might no longer use ZFS as its data storage.

~ ❯❯❯ docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: 1.12.1
Storage Driver: devicemapper
 Pool Name: docker-253:0-2753561-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 11.8 MB
 Data Space Total: 107.4 GB
 Data Space Available: 31.79 GB
 Metadata Space Used: 581.6 kB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.147 GB
...

The output information from docker info command confirmed my suspicion. But how do we change the storage back to ZFS instead of devicemapper? According to ZFS storage in practice the only prerequisite to have ZFS as the data storage is having /var/lib/docker as a ZFS dataset.

I was under the impression that tank/docker was mounted to /var/lib/docker. Actually /var/lib/docker directory lived on my local LVM file system (hence devicemapper was the storage driver).

To fix this, I stopped docker service, cleared out /var/lib/docker, and re-mounted the tank/docker dataset.

# systemctl stop docker
# rm -rf /var/lib/docker/*
# mount tank/docker
# systemctl start docker

Let’s see if it’s working again.

docker info
Containers: 6
 Running: 0
 Paused: 0
 Stopped: 6
Images: 18
Server Version: 1.12.1
Storage Driver: zfs
 Zpool: tank
 Zpool Health: ONLINE
 Parent Dataset: tank/docker
 Space Used By Parent: 1047826432
 Space Available: 263167705088
 Parent Quota: no
 Compression: off

# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
abiosoft/caddy      latest              af55a59400be        2 days ago          40.69 MB
...

Everything seemed to back to normal. I’m still not sure why tank/docker wasn’t mounted on boot, but I’ll leave it for another day. As for now, I’m quite happy.

Make Gnome Title Bar Small Again

Sometimes in life greater is not always better. Here’s a config which I used to make the Gnome 3 title bar smaller.

Create a file in ~/.config/gtk-3.0/gtk.css:

headerbar.default-decoration {
    padding-top: 3px;
    padding-bottom: 3px;
    min-height: 0px;
    font-size: 0.7em;
}

headerbar.default-decoration button.titlebutton {
    padding: 0px;
    min-height: 0px;
}

Here’s how the title bar looks like before and after applying the custom css file:

Default title bar size

Default title bar size

Small title bar

Small title bar

All credits go to the folks from these links:

Install Hack font on FreeBSD 10

FreeBSD 10 comes with a pkgng package manager which allows to install binary packages faster and easier.

Unfortunately, there is no binary package available for hack-font for FreeBSD 10. What is hack-font? It’s a very good font which looks very good for source code.

Here’s how to install it from source.

Check /usr/ports/x11-fonts to see if there is a directory named hack-font. If not, you might need to download or update the ports first. If you don’t have ports installed at all, do this:

# portsnap fetch
# portsnap extract

If you already have ports install previously, but just need to run the update:

# portsnap fetch update

Then navigate to /usr/ports/x11-fonts/hack-font, run make command as the following:

# cd /usr/ports/x11-fonts/hack-font
# make install clean

That’s it. If you use portmaster command to install package from source, you probably can just run:

# portmaster x11-fonts/hack-font

Enjoy hack-(font)ing!

Puppetserver failed to start after upgrading to 2.5.0

I upraded puppetserver from 2.4.0 to 2.5.0 on a server running Centos 6.8. Then, puppetserver no longer starts.

The log (I think it was puppetsrver-daemon.log) said:

Exception in thread "main" java.lang.IllegalArgumentException: Specified bootstrap config file does not exist: '/etc/puppetlabs/puppetserver/bootstrap.cfg'

As the error reported, /etc/puppetlabs/puppetserver/bootstrap.cfg was not there. The disappearance must have happened during the puppetserver upgrade. I can confirm this by looking at my backup directory and finding the boostrap.cfg file there: /srv/backup/etc/puppetlabs/puppetserver/bootstrap.cfg.

According to this issue #1247, the boostrap.cfg file is broken into 2 files and relocated to a different directories. This issue was raised sometimes in May, and the problem only happened on my server today in August. How is that possible? This should affect more people right? So far, I haven’t seen any one reported it yet.

I need to get puppetserver running again and fast. Here is a quick fix.

Edit /etc/sysconfig/puppetserver, and make change to BOOTSTRAP_CONFIG variable:

#BOOTSTRAP_CONFIG="/etc/puppetlabs/puppetserver/bootstrap.cfg"
BOOTSTRAP_CONFIG="/etc/puppetlabs/puppetserver/services.d/,${INSTALL_DIR}/config/services.d/"

Start puppetserver and it is working again (for now).

# service puppetserver start
Starting puppetserver:                                     [  OK  ]

How to remove a user password

Sure, you Linux gurus must be laughing at me to see this post. However, this is my 2nd time needing to remove a user’s password and I had to Google for it! So here is just a quick tip to mind your truly:

# passwd -d your_username

In case you’re wondering why I need to remove a password for an account, it’s for my 4 year old daughter who enjoys playing supertuxcart. She wants to have her own user account on a Fedora 24 box without a password.

Ref: https://ask.fedoraproject.org/en/question/47102/how-to-remove-user-password/

បង្កើត RSS feeds ពី SoundCloud និង iTunes urls

ខ្ញុំ​​ចូល​ចិត្ត​ស្តាប់ ផត់ខាស់​ណាស់ ពីព្រោះវាមាន​អត្ថប្រយោជន៍​ជាច្រើន។ តើផត់ខាស​បានការអ្វី​ខ្លះ? បើសិន​ជាអ្នក​ចង់ដឹង សូម​សួរ​ក្នុង​ខំម៉ិន​ខាង​ក្រោម។

ថ្ងៃ​នេះ ខ្មុំ​ចង់ប្រាប់​របៀបបង្កើត RSS feeds ពីផត់ខាស់​ដែល​ត្រូវការ អាយធូន​ដើម្បីស្តាប់បាន។ ជាមួយ RSS feeds យើង​អាច​ប្រើ​កម្មវិធី​ផ្សេងៗ​ដូចជា អាម៉ារ៉ុក។

មុន​ដំបូង​ យើងត្រូវ​ចំលងលិង អាយធូន ហើយ​ចូល​ទៅ http://getrssfeed.com/ និង​បិទលិងនោះ។ វិបសាយ​ getrssfeed.com និង​ផ្តល់​លិងថ្មី ដែល​ជា RSS feed.

វិធីខាងលើអាច​ប្រើជាមួយ​ SoundCloud urls ផងដែរ៕

Resizing an LVM logical volume

One of the logical volumes on my desktop is running out of space. I want to add 200GB to that volume. Here is one way to achieve this task.

First a bit of warning. Don’t, or never, just copy and paste what you see here if you want to apply it to your own system. You may accidentally destroy your precious data.

# lvs
  LV      VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  backup  data   -wi-ao---- 200.00g                                                    
  scratch data   -wi-ao----   1.00t                                                    
  home    system -wi-ao----  90.00g                                                    
  root    system -wi-ao---- 100.00g                                                    
  swap    system -wi-ao----  16.00g                                                    
  tmp     system -wi-a-----   4.00g                                                    
  var     system -wi-ao----  20.00g  

Let’s check the available disk space of volume groups we can use:

# vgs
  VG     #PV #LV #SN Attr   VSize   VFree  
  data     1   2   0 wz--n-   1.82t 639.01g
  system   1   5   0 wz--n- 232.38g   2.38g

I want to resize ‘scratch’, which sits on ‘data’ volume group. There are about 639GB free on ‘data’.
Let’s give ‘data’ another 200GB.

# lvresize -rL +200G /dev/data/scratch
  Size of logical volume data/scratch changed from 1.00 TiB (262144 extents) to 1.20 TiB (313344 extents).
  Logical volume scratch successfully resized.
resize2fs 1.42.13 (17-May-2015)
Filesystem at /dev/mapper/data-scratch is mounted on /srv/scratch; on-line resizing required
old_desc_blocks = 64, new_desc_blocks = 77
The filesystem on /dev/mapper/data-scratch is now 320864256 (4k) blocks long.

Very the change:

# lvs data
  LV      VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  backup  data   -wi-ao---- 200.00g                                                    
  scratch data   -wi-ao----   1.20t                                                     

Now ‘scratch’ has a total of 1.20TB.