ν42

Bleach your files faster!

Continuing from where left off in “Another Optimization Story”, seeking ever faster ways of mapping to and from whitespace characters … more

Another optimization story

Inspired by a post on encoding/decoding files to/from sequences of pure whitespace characters, I try to replace a couple of small lookup tables with simple computations … more

Perl’s File::Find on Windows: A path forward?

In Implementation by Wishful Thinking, I looked at a problem caused by a simple change to how File::Find handles symlink related options. The problem itself was moved several steps from the actual change in File::Find and manifested itself in Module::Pluggable not being able to find plugins. … more

Implementation by wishful thinking

It is natural to want the world to be simpler. To wish legacy code away. To want to avoid dealing with platform specific differences.

It is natural. But, in most cases, it also not possible for the world to be simpler. Legacy code is always there. And, there are unavoidable platform specific differences. … more

Who’s testing the tests? The case of a false negative

I typed cpanm JSON::MaybeXS and went to get coffee. When I got back, I reviewed the log file and noticed something curious:

Number found where operator expected at (eval 7) line 1, near "require threads::shared 1.21" (Do you need to predeclare require?) t\125_shared_boolean.t ..... skipped: no shared_clone)

Hmmm?! “Do you need to predeclare require?” I shouldn’t. It is a Perl builtin. … more

Some useful principles of unit testing

Much pontification exists when it comes to unit testing. Developers get introduced to the idea of TDD with simple code such as testing a method that adds two numbers etc so that they can learn the mechanics, but such motivators obscure away the very real tradeoffs that arise from the fact that tests are also code. … more

Use your own WiFi connection test server in Windows

Sometimes, things work for the wrong reasons. … more

UTF-8 everywhere and command line argument expansion on Windows

I want the utility I am writing to handle filenames containing non-OEM characters and have the benefit of file name expansion in command line arguments. … more

Upgrading from Intel 7260AC to MPE-AX3000H

It really is an improvement … more

On Windows, how do you get time in nanoseconds in C?

POSIX clock_gettime and *nix gettimeofday was not available when compiling C code on Windows using MS Visual C. Starting with Visual Studio 2015 (cl version 19.00), C11 standard timespec_get is available and provides the necessary functionality. … more

ATtiny85 chirping in response to red key presses on a Liyafy HC-35 keypad

Last time, I was just excited to get some lights to blink. Since then, I’ve figured out a few things. For example, if a bit in the output is set, the corresponding LED on the HC-35 is turned off. Therefore, to get the pleasing animated counting effect I wanted, I needed to send the complement of the number to the shift register. … I took for granted that I could read the keypresses using four wires. That turned out to be a correct assumption. So, four pins for reading keys, one pin for VCC, one pin for GND, the RESET pin which I am not going to disable because I do not want to “need a high voltage serial programmer to reprogram the chip”. That leaves one pin to get information out. … more

Fun with an ATtiny85, Liyafy HC-35 keypad with eight LEDs, and a serial to parallel shift register

My search for a reasonably constrained environment led me ATtiny85. I decided I wanted to do something with, but I did not know what. So, the first thing I was to wire up an ATtiny85 to an OLED screen and display a (now much smaller) set of rotating fortune cookies … more

Small is beautiful

When $WORK involves springing to life and destroying sets of infrastructure on AWS, you might want to go back to basics for a hobby … more

Don’t complicate things

I set out to simplify what seems like an unnecessarily complicated test … more

What is Perl 6 to Perl?

I was asked about what Perl 6 is to Perl. For example, is it an upgrade? Does it run existing Perl code? etc.

Earlier, I had been suckered into accepting the “sister language” narrative. Then, briefly, I thought Perl 6 was trying to be the Borg. Just a couple of days ago, it dawned on me: Perl is Omar Khayyam and Perl 6 is Hassan Sabbah … more

Another look at stock market behavior around "change" presidential elections in the U.S.

Due to a number of recent distractions, I totally missed the fact that more than 100 trading days have passed since the 2016 U.S. presidential election. Here is an updated look at the behavior of S&P 500 around change elections since 1952.

more

Data work is dirty work

Collating, tabulating, and graphing data is not analysis. Those comprise just the dirty work we must do to get to a point where it is possible to analyze data.

more

For your ’İ’s and ’ı’s only

Is there any way within the Unicode specification of preserving the identity 'ı' ≡ lc( uc 'ı' )?

more

Fixing Perl’s Unicode problems on the command line on Windows: A trilogy in N parts

I have used Perl on Windows for decades without being seriously hampered by any of its past or current limitations. Still, it would be nice to solve some of the issues, if only so I can post cute screenshots.

I’ve always thought I should make some effort to fix those some day. After I fixed Perl6’s Unicode issues on the command line in Windows, I felt slightly guilty that I had not given Perl the same TLC. Maybe “some day” has arrived. … more

Deception in tests considered harmful

I felt really good after finally ironing out all the kinks in PR #528 which added support to Perl 6 on MoarVM for handling Unicode script names, command line arguments, and environment strings on the Windows command line. Then I decided to run the stress tests … more

Notes on Unicode on the command line in Windows with applications to Perl and Perl 6

Handling of interesting characters on the command line in Windows or DOS environments has never been an annoyance free experience. Heck, 30 years ago, I was patching lookup tables in keyboard drivers for IBM PCs and compatibles at METU so we could write stuff using Turkish characters. At the time, there wasn’t even a standard Turkish keyboard layout. So, we have come a long way … more

\c[PERSON FROWNING, ZERO WIDTH JOINER, PROGRAMMER]

Going forward, I am going to occasionally try it out and, of course, write about my experiences because I enjoy trying things out and writing about my experiences. But, until Perl 6 developers accept input from people who are not members of their tribe more gracefully, I will stay away.

In the mean time, enjoy inserting emoji in your programs, but good luck if you need to capture output and test and exit code … more

Stock market behavior around "change" presidential elections in the U.S.

While writing yesterday’s post on the idea of selling the inauguration, I noticed that the behavior of S&P500 around President Obama’s election in 2008 was highly positively correlated with its behavior around the first time President Bush was elected in 2000. This is not something I had expected a priori, so I decided to take a closer look. more)

Does past behavior of SP500 indicate ‘selling the inauguration’ is a good idea?

A lot has happened since the last time I discussed a supposed crystall ball predicting an impending crash of the U.S. stock market. Today, I saw another chart which is being used to support the claim that:

Stocks tend to strengthen in the two weeks after Inauguration Day, but one-month returns are typically negative … more

Happiness is a mid-2010 13“ MacBook Pro with a new battery

For the past three or four years, it has been telling me that the battery needs to be replaced. Initially, the message was “Service Battery”. A couple of years later, it turned in to “Replace Now”. Recently, battery life fell to an annoying 90 minutes. … more

Adding USB3 ports to an older laptop

After the ancient replacement to my old Lenovo 3000 N100 died again, I got a Thinkpad T400 on eBay and transferred the hard drive to that one. At the time, I thought the T400 had two PC Card slots which limited my options for doing something useful with that expansion option. … more

Generate k random samples of n draws from a set of distributions in C++

In my quest to improve the tests in Boost.Accumulators, I decided to generate some random samples (i.e. sets of draws) from known distributions and compare the results produced by the P2 algorithm as implemented by a well-respected data analysis software package to the medians calculated using the Boost.Accumulators implementation. It turns out, the well-respected software does not implement the P2 algorithm in the way described in the original paper. It does something proprietary (which, of course, means that the algorithm they implement is no longer the P2 algorithm).

I am going to write in more detail about this later, but it’s depressing. In the mean time I decided the best way to cheer myself up is to write a quick C++ program to generate the pseudo-random samples I am going to use to compare the two implementations. While I have always liked C++ , it has recently become a lot more fun (assuming you have a large scrollback buffer in case you get any template related errors or warnings when compiling your code ;-) … more

A discrepancy between P-square algorithm implementation in Boost.Accumulators and the original paper

At the end of my previous post, I highlighted a discrepancy between the output produced by the Boost.Accumulators implementation of the P2 algorithm. It turns out the discrepancy was due to a typo in the original paper and not in the Boost.Accumulators implementation as I had originally suspected. … more

Know what you are testing: The case of the test for median in Boost.Accumulators C++ Library

A few days ago, I wrote about my adventures with the Boost.Accumulators library. Since then, I have spent some time staring at the test code for the median algorithms included with that library. The provided test demonstrates many problems created by flawed attempts at doing TDD. … more

Descriptive Stats with C++ and Boost.Accumulators

Boost is a collection of mostly header-only libraries which make mere mortals like myself free-ride on years of accumulated wisdom of talented C++ programmers. It is also rather straightforward to use it on Windows which is a benefit for people like me who does stuff on all three major operating systems.

One Boost library I hadn’t used at all was Boost.Accumulators. The other day, I was wondering how much work it would take to write a small program to calculate simple descriptive statistics on a column of numerical data. … more

Another Lenovo 3000 N100 bites the dust

It happened again.

My Lenovo 3000 N100 died.

This time, it happened while I was working on something. I was typing something, nothing particularly demanding was going on in the background, and, all of a sudden the screen went blank and all the indicator lights turned off. About a dozen different attempts at bringing it back to life did not pan out. … more

How to sum data from multiple files in Perl?

Stackoverflow is not that much fun any more, but I still look there to see if there are any interesting questions. Sometimes, relatively ordinary looking questions lead to useful insight. One such example is the question titled Perl : Adding 2 files line by line. … more

Manipulate %PATH% in the current Windows cmd.exe window with help from Perl

I often find myself working in one ConEmu window for extended periods of time. Sometimes, I end up adding a bunch of directories to the %PATH% during a session. And, more often than not, I end up wanting to just remove one or two directories from the %PATH% and continue working. For example, I might want to switch to using Cygwin’s git instead of the MinGW git installed alongside Visual Studio 2015.

So, I wrote a quick Perl script to do that. It outputs a string so I can grab that and set the path in the current shell. … more

An old Visual C printf bug affects Perl

Perl doesn’t yet build with Visual Studio 2015 tools out of the box on Windows. I have been using VS 2013 to build my perls on Windows 10. A few days ago, I noticed that a C program and an equivalent Perl program were printing floating point numbers differently … more

Define constants for use by other modules in Perl

We regularly encounter lists of constants. Quick examples are HTTP status codes, Virtual Key Codes, lists of error codes and many many more. Regardless of the language you are using, you need a way to use those constants via friendlier symbolic names rather than literal numbers … more

Display git branch in Windows command prompt

David’s post “Displaying the Git branch in the terminal prompt with and without Perl” reminded me of a batch file I use for the same purpose. Of course, vanilla cmd.exe does not allow dynamic updating of the prompt, so it is rather more limited, but it can still be useful if you spend most of your session within one repo … more

Perl is not the fastest language

Back in December 2015, I noted that perl was getting faster. Perl 5.24 was released a few days ago. The upgrade enabled perl to run the nbody benchmark 33% faster … more

Looking back at the scary DJIA crystal ball

Back in 2014, a chart, purporting to show great similarities in the movements of the DJIA at that time with the period preceding the crash of 1929, was making the rounds on Wall Street … more

MacGyvering HTML email using Perl’s Template Toolkit

Producing HTML email is generally a frustrating endeavor akin to traversing a minefield one step at a time.

Today, I started thinking about how to do this sort of thing without writing any code, without manipulating any trees etc etc.

I know … young hipsters these days probably have some gigabytes of Node infrastructure in their home directories with watchers, digesters, gulpers, notifiers and the like to do this kind of thing, but I have Perl and the Template Toolkit! … more

A tar anomaly

A bug-fix to File::Which caused me to discover some unexpected (by-design) behavior of MinGW and Cygwin versions of the tar utility on Windows: Files may evaporate on extraction … more

Perl is spreading to Windows

Installing git alongside Visual Studio 2015 Update 2 CTP brings perl 5.22.0 to Windows developers´ machines … more

Installing CommonMark Perl Module on Windows 10

I wanted to use cmark, the C reference implementation of CommonMark. Building the library and the command line utility using Visual Studio tools is straightforward enough. But, I ran into a few roadbumps on the Perl side of things … more

A first look at zsun WiFi SD card reader

zsun WiFi SD Card Reader, in a nutshell, is a WiFi access point with more flash (16MB) and RAM (64 MB) than most OpenWrt supported routers. Add the fact that you can plug in a rather sizeable micro SD card (I had a 16 GB one lying around, but I believe it takes up to 128 GB), and you have a matchbook sized computer with interesting possibilities … more

An explicit solution for d-digit excellent numbers in terms of factors of 10^d - 1

I got a lot of great feedback after posting about excellent numbers on Hacker News. An anonymous user, going by the nickname Someone, noticed a way to re-write the equation for excellent numbers. … more

An excellent optimization story

I was able shave about 95% from the running time of a long running program searching through large sets of integers. While the particular problem might sound esoteric, this type of thing pops up in many scientific and financial contexts where multi-precision arithmetic libraries tend to be heavily utilized. … more