Updating OpenWRT 19.07.6 and the way to solve the DHCP & DNS issue

The Linksys WRT1900ACS has been updated to the latest version OpenWRT 19.07.6 while the procedure was painful. Here is how to fix it. 

Since the OpenWRT 19.07 release, almost every minor releases (except the 19.07.4) has a serious bug that will hinder the DHCP and DNS functions. This bug can be observed every time just after updating to a minor release or sometimes just even by OPKG updating by SSH. The symptom is that the NAT function seems disabled and the internet is down. That's why I have been sticking to the old version 18.06 in 2020. 

Accidentally I found a way out: the solution is to switch to boot from another flash boot partition, reboot, and then switch back, reboot again --- that's it! 

It seems that rebooting this device even by switching off the hardware power button or unplugging the AC adapter is not a "fresh" reboot --- it still keeps some cache or something in the system and just doesn't flash out. The switching partition method sounds like a deep or clear-up reboot that removes all the incompatibility issues after upgrading. Since this bug was not reported widely and by other device owners, I highly suspect that this bug is specific to the Linksys routers with dual boot partitions.  

For detailed instructions for switching boot partitions please refer to Linksys Official Support - How to revert from OpenWRT to Stock Firmware (https://www.linksys.com/us/support-article?articleNum=316324). 

macOS Big Sur is buggy

Apple macOS Big Sur is the most disappointing OS version I've ever had. 

I've tried four versions of Big Sur (11.0 Beta, 11.0 official release, 11.1 Beta, and 11.1 official release) but a critical bug was still there and never fixed. Almost every day, the computer will reboot itself with critical system error warnings without any sign before many times. This critical error is driving me mad! So I went to the Apple Store to downgrade the system twice and stick to 10.15 Catalina. I'll check the security updates only and wait for the next major update of the OS. 

OpenWRT 19.07.4 fix DHCP & DNS issues

Linksys WRT1900ACS was updated to OpenWRT 19.07.4 this week, the fourth amendment to the major release of 19.07, which finally fixed the issue of DHCP and DNS. This version looks stable but the Luci web UI responses noticeably slow than version 18.06. 

Anyway, this version is recommended also for a considerable number of security fixes. 

The ageing hard disks

Rebuilding an array of RAID disks can be time-consuming and also brings more harm to the old disks alongside. Just after a month of the death of the first WD40EZRZ, another disk manufactured on the same date was reported by S.M.A.R.T with C5 and C6 errors and the raw values were "1" in two entries. Luckily, this bad sector is not spreading and the bad count remains the same after a few days. So in about a month, I bought 4 HGST HDDs one by one to replace the old ones. 

What a nostalgic brand and product line! Rebranded as a high-end product of Western Digital, the UltraStar is now the only one survived, while the TravelStar and DeskStar were removed years ago. Everything has changed all these years, but never the random reading noise of this brand! This identical noise comes from the old IBM HDD department, the producer of the first hard drive ever. The IBM/Hitachi/HGST is the name for reliability. The oldest working HDD in my hand is an IBM TravelStar DARA2600 6GB IDE 2.5-inch. 

The first was a brand new Western Digital UltraStar HC320 HUS728T8TALE6L4. It is perfect but expensive. The later were three used HGST UltraStar HUS724040ALE641 HDDs. They are cheap and of good condition --- but ageing obviously. 

The three disks were of the best class in production, as the brand new 8TB disk, of course. The metal cover was finely finished and reading sound is still strong and good. However, their SMART info shows some noncritical but still worrying indicators, e.g. read error rate, throughput performance, and seek time performance. Hope they will survive longer than 2 years. 

Remagnetising bad sectors in WD40EZRZ

I have an HP MicroServer Gen8 with a 12TB Windows Storage Spaces parity volume based on four WD40EZRZ disks. This week, one of the drives was reported by CrystalDiskInfo (https://crystalmark.info/en/software/crystaldiskinfo) with S.M.A.R.T "C5 Current Pending Sector Count" and "C6 Uncorrectable Sector Count" cautions. The C5 and C6 entries were highlighted by the tool but without any specific raw data reported, which left the damage details unknown. 

The malfunctioning disk was a WD Green, a low-end 540RPM power-saving product discontinued in 2017 and then rebranded in the WD Blue 5400RPM name. This product line was designed for cold data archive, so it is a little over-demanded in a NAS array environment. The disk was produced in mid-2015 and has been running for over 10,000 hours, especially was not stopped in the last 6 months because I could not touch it due to the COVID-19 social isolation. 

In the last few weeks, there were uncommon noises and disk I/O delays, so I was checking S.M.A.R.T info more frequently. As the warnings were shown yesterday, I used WD Data LifeGuard Diagnostics to run a Quick Test but it failed to complete the scan very quickly. 

The symptom was not new to my Western Digital products. In 2013 two 2.5-inch drives WD5000BEVT and WD1200BEVS had the same problems and was repaired successfully by HDD Regenerator 1.71. The two drives are still working properly today but not used to store any important data. According to HDD Regenerator's website (http://www.dposoft.net/hdd.html), the mechanism of this tool is: 

Almost 60% of all hard drives damaged with bad sectors have an incorrectly magnetized disk surface. We have developed an algorithm which is used to repair damaged disk surfaces. This technology is hardware independent, it supports many types of hard drives and repairs damage that even low-level disk formatting cannot repair. As a result, previously unreadable information will be restored. 

So I decided to repair the bad sectors with my old tool but met more difficulties than I had thought. The HDD Regenerator 1.71 bootable USB disk was too old to boot on my Gen8 system, so it took me one day to find a 2011 version and some tricks to create a useable USB disk. During the process, I almost gave up for some times and used some alternatives but they are just rubbish if compared with HDD Regenerator, even if the latest version was released exactly 10 years ago --- kudos to Dmitriy Primochenko!

The repairing process has started this evening and the damage was much worse than I had expected. With just 1.58% of sectors scanned, there were already 92 sectors found damaged, compared to just one to two bad sectors in each drive in the 2013 cases. The software is estimated to scan and repair for over 18 hours. I'm also worried about whether it can work properly after the repair because I saw bad sectors spread in some previously healthy sectors. 

In the end, the disk did not go through it. After backing up all the data, the bad sectors spread considerably and it was even not recognised in the system. This old drive is completely gone. Fortunately, there is no data loss thanks to Windows Storage Spaces parity technology and CrystalDiskInfo's early warning. 

A new drive has been booked and will be shipped soon. 

The unstable OpenWRT 19.07 DHCP & DNS

The latest versions of OpenWRT, i.e. 19.07.1, 19.07.2 and 19.07.03, are unstable with the LAN DHCP and DNS functions. This issues can also be introduced in OpenWRT 18.06 by upgrading to the latest OPKG packages related to DNS or PPP, but the specific mechanism was unknown. 

After upgrading the symptom was: 

1) unable to DHCP new IP addresses to new connected devices in the LAN

2) unable to ping the router in the LAN;

3) unable to resolve IP addresses of WAN websites. 

The router was forced to downgrade to OpenWRT 18.06.8. The downgrade process was painful and requires a fresh reset to the factory default settings fo OpenWRT, which means the WiFi signal was disabled in the first place. 

Bose QC35 II: A two-year review

I bought the first Bose QC35 II as a gift for my dad at the end of 2017. The active noise cancelling (ANC) was out of expectation and the sound quality good enough as a Bluetooth headset. So I bought a second one for myself and was using it extremely heavily from March 2018 to Jan. 2020 when the right unit is not working anymore. 

This post is a two-year review of this product based on my personal experience. 

An ANC headset is a highly recommended gadget and almost cannot be get rid of since the first day of using it. It is a useful tool to block out the noise in a metro coach or of the fan noise of a computer/air conditioner. Given my over 5 hours in metro and over 50 hours in front of a computer each week, the headset contributed greatly to my working proficiency. I was wearing it for more than 16 hours per day, so the total working hours of this device should be as equal to average users' four to five years considering my extremely heavy use. For the last six months, I was wearing it all my sleep time. And for about half of the time, I was wearing it as over-the-head earmuffs without playing any music. It is a strong workhorse with reliable quality during its lifespan. 

Another unavoidable topic about the Bose QC35 II is its competition against the SONY WH-1000X M3. Very fortunately, I was trying a WH1000X for nearly a week but returned it in the end. Frankly speaking, there are some fields that SONY outcompetes Bose, e.g. sound quality, build quality, and even the ANC level. By looking at the specifications, SONY should be the absolute champion in the market, but actually, the Bose products are better in the features that cannot be measured by numbers, e.g. button feelings, Bluetooth connection stability, and the comfortableness of wearing it. Especially WH-1000X has a mysterious interface response when connecting/disconnecting and switching among devices. The Bose headset is extremely good at connecting to TWO dives at the same time, and I don't have to switch between devices; while the SONY headset even doesn't have a sound signal when connecting/disconnecting, which is barely acceptable. 

Last but not the least, the firmware issue. It is interesting to see that the ANC performance is not always at its best level after some versions of firmware installed. Particularly the most criticised 4.5.2 version which was released in mid-2019 but has not been updated since then for half a year. Though similar cases were being reported for years, it is the first time for some YouTube uploaders to use spectrum devices giving objective testing curves. Users were discussing on the Bose Community forum the way to downgrade to some earlier versions but it does not always work possibly due to the hardware version differences (AZ or AE). What's more, compared with the slow movement in updating the firmware, the Bose was extremely quick at deleting any posts about downgrading methods and eliminate the downgrading possibility on the server end. This is very disappointing indeed. 

Generally speaking, Bose QC35 II provides the best ANC technology now but the build quality and after-sale service should be improved.