articles

Stylometry: A Bird's Eye View

Now that large language models (LLMs) have become part and parcel of how we conduct text authoring, there is a danger that this will result in a loss of linguistical diversity in style as more and more people lean on LLMs to expand on their ideas. I'll readily admit that …

Where TDD Falls Short

TDD is not design.That's it.While TDD does produce robust code, it amounts to building with a scaffold. The only problem is that the scaffold ends up being the life support for the application.While the test suite then serves as an aid to prevent regression of the code, …

A4risms

“The antidote for arrogance is service.” [2024-03-01]“There exists an academic paper which ‘supports’ virtually any position.” [2024-02-12]“Most people in software are oblivious to the fact the ‘Agile’ is an aspiration not a capability.” [2024-02-11]“Leadership is not about giving people what they want; it’s about giving people what …

Vectors: The Danger of A Bad Definition

"A bad definition is worse than useless––it is misleading and the cause of endless confusion."The standard definition of a vector is: "a way to describe a quantity that has both magnitude and direction". Even ChatGPT4 states that: "A vector is a mathematical entity that possesses both …

Abstract vs Concrete Knowledge

I have a growing realisation of a noxious temptation to want to know more than I need at any one moment. Recently, I've found myself veering towards exotic mathematical ideas I was exposed to as part of my engineering education. For example, I was recently going through Maxwell's equations that …

Strategy

I've come across the word 'strategy' quite a bit over the last few weeks and it just got me thinking: what does it mean? How is strategy different from other conventional thinking?I tend to think of strategy in the context of chess. Strategy is often pitted against tactics. …

Effective Writing from Larry McEnerny

If you have not watched this inimitable talk on writing then you must see it here and another version here.The quality of writing is in the eye of the reader. There is no such things as good writing objectively.Ensure that there is a consistency in the …

The Vision of Software

“To allow any user to easily complete domain tasks in such a way as to be indistinguishable from a domain expert while enforcing domain standards.”

Unnaturally Agile

Inasmuch as the Internet has become a treasure trove of information, there is one form of information that abhors its radical transparency: curated content. The best it offers in its place is crowd-curated content or social knowledge such as on StackOverflow and its myriad sibling sites. Even with social …

Data Science Value

The value of data science is two-fold:insights through the interpretation of statistical models, orinsights through prediction from statistical models.As with any value, the true measure is time-to-value: how long does it take to get insights? We would much rather get value sooner rather than later. However, we …

The Government Superstore

Suppose you read a announcement that the government would like to take over the operation of the largest supermarkets with a view of ensuring quality and affordable prices. "In the wake of the current economic woes," it read, "we believe that the public will be assured of higher quality and …

Generations Do Not Exist

I think the notion that human populations exist in generations is fallacious because it suffers from a subjective bias. Any individual who makes a claim contingent on the idea of a generation automatically thinks in terms of the distinction between his age group, his parents' age group and his offspring's …

Step 1: Establish Control Limits

According to Deming, management is not playing games; it is all about prediction. A manager must be able to predict with a fair degree of certainty the likelihood of certain outcomes. How can this ever be achieved? It stands to reason that the first principle of prediction is knowledge of …

Popular Science is not Science

What role does popular science play in society? Well, I've woken up this morning thinking about electromagnetic field theory and how this is extended to field theories in general. There is not rosy way of explaining it. Yet, there are numerous popular science books and documentaries, about physics in particular …

Reactive Programming Resources

I've just completed the O'Reilly course on Reactive Python for Data Science, a wonderful course only hampered by the fact that it is based on an outdated API. I'm still coming to grips with the key concepts because it's all so new. I'll list below (for my future self) …

Principles of Autonomy

In order to realise true autonomy, a system will require to abide by the following principles:Existence of state - the system must exist in distinct states with clear state transitionsAwareness of state - the system should know the current state; a system should be capable of self-observationDecoupling …

Sequence Clarity Requires Search

I now base most of my thinking on data upon the following premise: data consists in only two forms - sequences or measures. Sequences are assemblies of symbols defined a priori by some social convention to convey meaning while measures are numerical estimates (whole or fractiona) of the attributes …

Feature Emergence

"Features are always and everywhere aspects of integration, not implementation."---Righting Software, Juval LöwyWhen I was a graduate student, I once ventured to acquire a toy helicopter for two reasons: 1) I could finally afford such seemingly useless distractions and 2) I had always been curious about toys that …

Trust ChatGPT4?

How to hire a bioinformatician

I recently received a request from a friend to advise him on how to hire a bioinformatician. Having been through an interview process quite recently, I thought I'd put this here as a record of what I believe is a useful approach to hire at the right skill level.By …

The Physics of Software

This is a work in progress. It will undergo continual revision.There is a fundamental flaw in how we conceive software, which has drastic effects on how we work with it. As its name suggests, the name 'software' implies that software is intangible. This is not correct. I would like …

Scientific Prioritisation

Non-lineary is everywhere. Faults in a machine are not uniformly distributed but are interdependent on one another leading to a power relation or even an exponential relation. This is the case in virtually every system. The solution to addressing these problems is to prioritise the order in which problems will …

Software Production Using Just-In-Time

Written on a phone. Expect typos. I’m presently reading Taiichi Ohno’s inimitable Toyota Production System: Beyond Large Scale Production, which is such a pleasant read. It’s crisp and concise with unique insights into the development of the keys ideas of TPS. It also serves as a poignant historic of …

CPUs: Think Engines Not Brains

Computer processors (CPUs) are not brains and are in no way similar to them. I hear it time and time again and I can see why it is such a pervasie non-truth. However, repetition is not an avenue to truth.There is a direct analogue between Charles Babbage's analytical engine …

The Volatility Argument for User-Oriented Design

User interfaces are the points of the highest variability in usage. Therefore, the most prudent approach to design is to exhaustively explore the user space and thereby present as natural and simple as possible a set of interfaces which mediate the intended use. This may even involve inventing new types …

Class Diagram Relations

I'm only putting this here because I will have to refer to it repeatedly in the future. There is no way I'll memorise this.

NLI: The Future of Usability

One of the most enduring ways to present software tools is through command line interfaces (CLIs). They are the bread-and-butter of most scientific computing and core software tools such as Git are built around them. CLIs have, over the years, converged on a pseudogrammatical style consisting of the following components: …

From IT to KT

When Claude Shannon came up with his treatise on information theory, the one thing that was absent from his mind was the notion of semantic information. Information, he surmised, is in the eye of the beholder. The resulting technological revolution has always been grounded on these presuppositions: the utility of …

DNA: On Another Level

"By information I mean the specification of the amino acid sequence in the protein... Information means here the precise determination of sequence, either of bases in the nucleic acid or in amino acid residues in the protein."- Francis CrickYesterday, I got hit by a bus. A big, fat, …

Data Science vs. Statistics

Data science has become such a promiment field that it is almost impossible to get an answer that resonates across multiple sources. Typically, there will be a mention of statistics, data, coding, machine learning and, quite often, AI but occasionally sources toss in 'scientific computing' and 'the …

A Taxonomy of Data

When we talk about data, we have in mind the raw digital representation associated with one or more events. This we must distinguish from knowledge itself and information, which I will not attempt to define here.There are three main types of data:Media.Measurements.Sequences.The type of data …

Too Many Options Are Unusable

Two examples: ImageMagick and iTermHere are ImageMagick's `convert` options:convert -h Version: ImageMagick 7.1.1-11 Q16-HDRI x86_64 21206 https://imagemagick.org Copyright: (C) 1999 ImageMagick Studio LLC License: https://imagemagick.org/script/license.php Features: Cipher DPC HDRI Modules OpenMP(5.0) Delegates (built-in): bzlib fontconfig freetype gslib heic jng jp2 jpeg jxl lcms lqr ltdl lzma openexr png …

Why I'm on the fence about cloud computing

I've always wanted to run my own servers. The first 'server' I bought was a €20 PC with 2GB RAM, which I attached to my home network and set up as a webserver. I was able to SSH home from work and fool about but did not do anything more …

Lessons from Djokovic's 2023 Wimbledon Loss

The fate of a game is decided on the practice court. When the players step onto the court they are not merely availing their raw talent, however awesome that is. Instead, they are there to present the effectiveness of their practice sessions to overcome the competitor's practice sessions. At the …

Designing for Failure

By far, the biggest way to design for failure is in the social dynamics of the team. Broken teams, where personal egos are an essential production component, are by far the fastest way to crash. It is painful to be in a team where the structure of the team precludes …

False Consensus

Any situation in which there is a preponderance of positive affirmations while maintaining the exclusion of criticism inevitably leads to false consensuses. False consensuses are the renunciation of truth.If a society foster true freedom then false consensuses are unlikely to exist. For example, free markets allow for both profits …

Alphabet (Google) Broke My Heart... Again!

It's not news that Alphabet (Google's parent company) has, time and time again, flogged some of its services. In fact, there is a graveyard of services it has left in its wake. I suppose this is all part of its entrepreneurial pursuits to develop new services. The problem is …

Engineering Software

My thoughts on software are taking a new direction. I now think that producing high quality software involves three key stages:DesignEngineeringOperations, which can further be broken down into Production Operations and Support OperationsDesignThe design stage results in a design---an architectural representation of the proposed system …

Time-based System Effects

This is something new that’s just occurred to me. Typically we are used to thinking of systems as static entities whereby all components are present simultaneously in space time hence the notion of stasis. However, it is possible that system components may occur temporally so that the systems effect is …

The Value of Human Attention Has Just Shot Up…

1996 was a seminal year. It was the year that I began life away from home. This was the year that I began high school. As you can imagine, I was as excited and my mid mind filled with all the possibilities of what life away from home would be. …

Software Production: Think 'Stages' Rather Than 'Cycle'

I have previously written about the myth of the software life cycle. My problem is the use of the term 'cycle' implying a return to the beginning. While I stand by that article, I am aware of the progression of a software project through various stages. Therefore, it should …

The Subtlety of Software

Software breeds a pervasive subtlety that accommodates mediocrity in spades. This subtlety makes its presence felt by the fact that working software can be fashioned along a wide spectrum of quality. But given that quaity is a cost that must be borne by the producer of the software, the average …

Awareness versus Knowledge

It goes without saying that there is a difference between being aware and actually knowing that something is the case. What must be said, though, is that the preponderance of information in its various forms has made us lean more towards awareness than knowledge; that is, we are far more …

How to Build A World-Class Team

It is the responsibility of leaders to build world-class teams. There are two principle features of a world-class team:productivityprocessTo build a such a team we must go through the following stages:define the aimoutline the system to accomplish the aimspecify the process/methodology by which to …

This Impenetrable Cloud

I have a growing dissatisfaction with search engines that I have resorted to relying on published curated content (e.g. books) for reliable knowledge. Increasingly, I find that search engines do not satisfy my queries though I’ll readily admit that I’ve resorted to reading quora pages, which are more attuned to …

Team Life Cycle

Teams are social structures that like any other experience various stages of being. Just as we are born, go through various stages of development into adulthood, mature in our contribution to the world then eventually retire and die, so do teams and the tasks they go through. If teams perpetually …

Effective Intelligence

When we think of ordinary intelligence we are bound to think of the intelligence quotient (IQ). The IQ is a measure of ‘brain power’—your capacity to extract form from perception, usually visual perception. It is a power measure because it factors in time. Therefore, an individual with a high IQ …

On Capitalism

Capitalism is synonymous with money or credit. That is wrong. Capitalism has to do with an abundance of valuable resource under exclusive control. Money or credit is not a valuable resource—merely the capacity to complete transactions. Therefore, it is possible to have a capitalistic economy without any money …

On Productivity: Take 2 - Productivity As A Systems Effect

I have previously written about productivity. However, I feel that the list of points I outlined were all incrementally influential on productivity--you can only make superficial and volition-based gains in productivity. Instead, what is needed is to think of productivity, not as a volitional attribute (he is productive), but …

Negative System Effects

Systems benefit from the synergic effect of the components working together. By contrast, when components work against each other the result is work that is not the simple sum of their individual effects and may nigh be negative. Another way to think about it is that systems amplify the result …

Architects Define Boundaries

It is the responsibility of a system architect to discover meaningful boundaries that fulfil the expressed use cases. This elevates the place of use cases as more or fewer use cases may significantly alter the system boundaries. I believe the right way to then think about the system architecture is …

Good Teaching Is An Expression of Systems Thinking

A system is an assembly of components with effects outside the scope of any one component. These effects are due to interactions between components. This is why we think of systems as synergistic entities with emergent properties. Anything worth devoting significant time to learn is likely operating under system …

Systems and Apparent Complexity

Consider a girl at a piano. The way her fingers move is hard to associate with the beauty of what she produces (if she’s any good; let’s assume she is). Of course, she may convey something of the rhythm of the music by how she moves parts of her body. …

I Was Wrong: Exponential Gains Require Systems Thinking

I previously wrote about how exponential gains require exponential effort. I'm now coming to realise that this is not strictly true.To be precise, linear thinking results in linear output--there is no doubt about that. However, I'm developing a growing realisation that systems thinking i.e. capturing the whole …

Why Good Design Is Incredibly Difficult to Achieve

The act of design is contranatural: the second law of thermodynamics tells us why. Entropy serves to erode design and there is nothing in nature other than the persistence of life which is preservative or the act of will which is creative that oppose entropy. While we extol the …

Think Integration, Not Implementation

"Features are always and everywhere aspects of integration, not implementation."---Righting Software, Juval LöwyMy love for architecture has grown even more refined following the above description. The idea that designing against a feature spec is just ludicrous is one of the most counterintuitive ideas I've come across lately. It …

What's Not Special About 'Special Offers'

Special offers are not simple. While their intention is to make the proposition attractive to new clients, I find them very confusing especially if the intention is to establish a long-term business relationship. I believe that there are very particular cases where a special offer makes sense e.g. if it …

Software is Problem Solving Media

Media is designed to deliver some experience. Software is a special form of media with the goal of delivering a solution experience. In other words, unlike traditional passive media, software is active media for solving specific problems. That’s all there is to it!

The Achilles Heel of Formal Education

Universities are in the business of canonicalisation: they want to turn the exoticness of a career into a recipe for training thousands of young minds and in the process pocket the proceeds. There’s just one major problem with that: most illustrious careers are actually a hack resulting from a …

The Essence of Business Success

Success in business is predominantly the result of two tasks: outbidding and undercutting rivals for the same resources. Outbidding determines its capacity to consume resources and undercutting its capacity to conclude successful sales. While customer service and quality are very important, these are lagging measures. That is, it is far …

Test Coverage Should Be About State Space Not LOC

I've stated previously that test coverage by measuring lines of code (LOC) is not a reliable measure of code quality. The gold standard should be coverage of the state space. Most applications can completely be defined by a finite state machine: given a range of input parameters, the application can …

The Coupling of Information Technology Breakthroughs

It would be wonderful if hard drives could store hundreds of petabytes of data without a change in the size of disks. However, such changes would means corresponding exponential improvements in processors to clock data in and out, memory bandwidths and networks. It seems to me that the limitations we …

The Impartiality of Scientific Research

The scientific method's strength is its ability to lead to conclusions objectively. However, the immediate consequence of this is that by avoiding subjectivity it unwittingly assigns equal significance to every result. This is not to say that all results have the same value; rather, by not advocating any particular …

Simplicity is the Highest Form of Quality

I've been thinking about which measure of quality excels above all the rest and I can't help leaning towards simplicity. Of course, if a system is error-prone yet simple, it's not quite as simple as we expect it to be. Therefore, simplicity only counts when we are dealing with clean …

Design, Development and Testing Decoupled

The job title 'software tester' suggests the economic benefit of deferring software testing:after development is concluded, to a third party, not the person who wrote the code, andat the interface level.I've noticed that this job has grown in popularity and that it often does not require …

Great Quotes

“If you don’t at least occasionally contradict yourself, your position is not nearly complex enough.” - Terence McKenna

Paid to Think

Are you paid to think? Does your employer have you on the payroll, not just for your manual participation in some process, but for creative and original ideas? Then you are a knowledge worker in the pure sense. Knowledge work is nonlinear by design. It is impossible to guarantee a …

Django and Application State

Ever since I started writing applications in Django, one of the things that has perennially troubled me is the lack of awareness that Django apps have for application state. There is no interface by which a developer can express application state. While the framework does a remarkable job of capturing …

The Principle of Interfaces

Interfaces are predicated on the following idea: the number of items that may be interacted with to effect some effect is a tiny fraction of the number and manner of ways in which that effect may be produced. Interfaces serve as a way to drastically reduce complexity thereby hiding …

Entrepreneurship Is A Bid for Scarce Resources

The Garden of Eden was a system for the production and distribution of goods and services, but it was not an economy, because everything was available in unlimited abundance. Without scarcity, there is no need to economize– and therefore no economics.Thomas SowellEvery new business implicitly makes the claim …

The Trinity of Software

There are three essential elements of software:TypesOperationsConstructsEverything else in software can be reduced to these three. We use types to construct models, operations to act on and between models and constructs to organise execution. Types may be provided by the language though these tend to be …

Peer-Reviewed Data Archives

Popular scientific data archives eventually run out of resources to curate their data. It is much more practical to shift to a peer review mode for data curation. This will require high quality tools which enable the scientific community to assess data quality and validate the data. It may also …

Exponential Gains Require Exponential Effort

Update: I've a new perspective on this...Simplification i.e. the process of reducing complexity, exacts a non-linear cost because the process of identifying complexity-reducting factors is complex. If it were not so then the substrate would not be complex. Simple substrates require linear effort in order to reduce them. My …

Adobe Failing to Read the Writing on the Wall

Adobe has been the darling of the creative industry for decades but it's insistence on it's cloud-only package model may soon result in its demise. The fact that people literally beg and cry to get it at a more affordable price is a sign that it has managed to establish …

Does Object Oriented Programming Boost Productivity?

When done right, object oriented programming (OOP) converts the imperative nature of procedural programming into a set of declarative entities with imperative calls within and between them. The advantages of better reasonability, reusability and all the others depend on this. However, I am yet to meet someone who is able …

The Social Effects of Quality

When you take the time to deliver high quality your users respond with open hearts. They cheer you on. They are more than willing to give back to you considering the convenience you have afforded them. The world is awash with base attempts that promise heaven and fail to even …

HPC: Excessive Inventory

The trend for most scientific computations to be run on clusters or grids hides the need for improvements in computational design. Using high performance computing (HPC) resources typically involves providing prodigious amounts of compute and memory (and by extension networking). It is inevitable that the demand for storage should increase …

Unmanageable

Some tasks are unmanageable. Their inherent complexity renders external accountability impossible because the level of understanding required is just as complex as carrying out the task as to make it too expensive to manage. Such a relationship requires trust from the manager and honesty from the executor. For example, scientific …

The Five Sources of Wealth

As far as I can tell, there are only five sources of wealth:InheritanceLuckCreativity (not mere imagination)Talent (at a world class standard)Theft (like most politicians in poor countries)Only one is within most peoples control and not subject to sudden loss: creativity.

A Startup Is A Promise

Every startup is a promise that the new product or service on offer will be here to stay at a quality level that meet expectations. The novelty of the startup allows its potential clients to give it a chance to fulfil this promise. This only last for so long after …

Capital Threshold

I've tried searching on Google for this but apparently it's not a thing. The idea of a capital threshold is the minimum required capital required for an investment to be profitable. It applies to different economic jurisdictions implying that different economies incurr different capital costs to establish enterprises. Ideally, an …

Advantages of A Mature System

A systems can be said to be mature when its quality output is optimised under the given inputs. This means that one would very rarely observe special causes as these have been driven out of the system due to a systematic attempt to address all such causes. In addition, because …

Using Database Views with Django

Here is an excellent article which shows just how to do that: https://rescale.com/using-database-views-in-django-orm/The objective is to have read-only, analysis database views for derivative data that is based on aggregation of the actual data tables. I think this should be best practice and probably be natively supported from Django.

More On Management

I've previously written about management. While dwelling on W. Edwards Deming's ideas about the system of profound knowledge I can't help but conclude that management is the art of improving a system by coordinating the activities of a team towards continous improvement. One of the elements of the system of …

Entrepreneurship A La Deming

According to W. Edwards Deming, for an organisation to succeed it needs to incorporate a system of profound knowledge consisting of:Appreciation for a systemA theory of knowledge Understanding of variationPsychologyThis system of profound knowledge is a means for acquiring knowledge external to the organisation to become …

Design vs Engineering

Design has to do with human requirements. Engineering has to do with technological possibilities.

Competition vs Cooperation

Competitionis for losers - Peter Thielnobody wins in a price warwhen producers are forced to compete the divert attention from innovation to survival is pessimistic is a zero-sum game Cooperation makes for a bigger pie - better a smaller piece of a bigger pie than a big …

Interfaces and Models

We all want happy customers but more than that we want high quality even when the customers are yet to beat down our door. There are two avenues where quality lives: front office and back office. Your customers never see your back office but they feel its effects through the …

Execution Marks the Difference

Everyone had great ideas. But great ideas are really not worth much on their own. True value lies in the capacity to exploit great ideas. Great countries are those that live on the cutting edge by executing on great ideas. Great companies are those that harness great ideas, either internally …

Software as Media

In The Software Studio: A Software Production Process, I proposed the idea that we should think of the software production process in much the same way that movies are produced: with distinct phases through which the production proceeds. The more I think about it the more I realise how …

Designing with Eternity in Mind

The purpose of analysis is to discover the right design. It is possible to successfully execute the wrong design. When this happens, any success will have an expiry date on it because the inevitability of change will expose the inadequacy of the design. Design really is a life and death …

Intelligence: The Goldilocks Zone

It is possible to be too intelligent for your own good. When one is too intelligent they tend to be deployed where others can make the best use of them. This typically means that they will end up working in isolation on things that others are incapable of working on …

The Third (And Perhaps Most Important) User

I've always assumed that software should be written for two principal users:the end user in the domain, andthe programming user writing applications for the domain. But it's just occurred to me that in numerous cases there is a third implied user: the administrative user. In many cases …

DDD?

Yes, DDD! It stands for documentation-driven design. The goal is to have the documentation capture the domain usage for the respective users. Typically, documentation is written for the end, development and administrative users. Therefore, if the respective users find the documentation meaningful in the absence of running code then …

What does it take to create?

Over the last few days, I've struggled mightily to concentrate on getting my work done. This is mainly because we have a system that needs a lot of babysitting and this means that every single minute there is an email or Slack message vying for my precious attention. I find …

Monte Carlo Thinking

Sometimes it's hard to be creative and your only solution is to resort to randomness. By traversing random data you can prime yourself into expanding the scope of your thought. This is Monte Carlo thinking. The name is obviously inspired from the corresponding statistical technique typically applied in Bayesian modelling. …

Decouple Yourself!

In an earlier post titled "Show Me Your Hands!", I discussed that a great ideal to work towards is to be able to walk away from your code and still have it operate as it should. Your colleagues should be able to carry on where you left off without …

On Working with 3D Data

The ideas are straightforward; the problem is how these ideas are expressed through various programming APIs.The cast of characters: space handedness, image coordinate systemsDiscrete vs continuous spacesDomain issues: MRC filesGranularityConsider an arbitrary 3D dataset. We rarely think of 3D data apart from some space and …

"Ready, Fire, Aim!"

"Ready, fire, aim. Do it! Make it happen! Action counts. No one ever sat their way to success."Tom PetersSometimes the exhilaration of solving a problem is so exciting and pressing that there is no design to design. You would like to see a solution now - however ugly. …

Ensure PyPI Publish Action Does Not Fail Because of Wrong Version

Here is the scenario: you run a Github action which automatically publishes to PyPI on new tags (I'm yet to figure out how to publish on push only). Since there are two places to label the new release (the tag and the setup.py file), it is very easy to find …

Change Is Your Biggest Risk

Change is the law of life. And those who look only to the past or present are certain to miss the future.John F. KennedyIf there is any guarantee that any project should anticipate it is CHANGE. You can be 100% sure that something will change. You might not …

Runtime

Every computation occurs in some runtime. The challenge is to accomplish complex tasks using available runtime even when it is clearly impossible to use a single runtime to complete the task. Therefore, there is a need to atomise the complete task into a coherent sequence of runtimes to accomplish the …

Simple Communication Between Threads In Python

I've just discovered the threading.Event class, which has the methods set() and is_set() to communicate between threads. There is also a wait(timeout=None) method, which blocks until the flag is set.import os import sys import threading import time # this is where the real work is done def function(duration, e): …

Management Is…

Process development, maintenance and optimisation.

The Software Studio: A Software Production Process

"The Fog Creek approach really is that programmers are talent... the programmers here are like the stars and the directors of a movie." Joel Spolsky on Fog Creek SoftwareHaving worked over the last few years on a relatively unstructured project, I have a growing angst to elaborate a clear …

Producing Software: The Myth of the Life Cycle

Every time I read about how to produce software I invariably come upon the phrase 'software development life cycle'. What is cyclical about it? It's confusing and should not be referred to this way. It's almost as if once software is delivered the production team go back to the begin …

Code is Movement, Architecture is Direction

There is a recurring theme in my thoughts at the moment. I've been thinking about what it would take to sign up on one of the freelancer sites like upwork or freelance or fiverr. The thought of making an extra buck with my 'extra' time (I don't really have any …

Writing Quality is Domain-Bound

There is no such thing as good writing apart from a domain. Writing is thought expressed and for it to be excellent in and of itself, divorced from any domain, is ludicrous. This also means that, though one may be regarded as a prodigious and proficient writer in some …

Coding as Exploration

Consider this article as a brain dump. One day it will be refined..."Software is not building. It's not construction. It's design. And design is like exploring a space of possibilities and what all the possibilities are, and you don't know how long it's going to take. It's like science. …

Teamplayer?

This is a raw dump of a swirl of ideas running through my mind. If they are incoherent it is because I'm still in the process of organising them.The vision of teamwork is accomplishing more than what a single individual can accomplish.If you're like me, it is rare …

Make Change A Core Feature

You haven’t designed well if you haven’t accounted for change. You can spend ages designing the best software but everything can be thrown down the toilet with a simple unanticipated changes. This is the reason I have a growing respect for the idea of emergent design.

Draft CLI Architecture

Here is a draft architecture that I have been working with for CLI applications. It is not complete because it has some holes in it. The whole idea is that it is built will layers each of which enjoys the services of the layers below it. Personally, I think there …

How to Move Fast in Software

"Take time for all things: great haste makes great waste."Benjamin FranklinWho wouldn't want all their features implemented immediately? It depends. How long will this software be used for? If it will be single-run code then nobody cares because it will not need any maintenance and we can always …

Think Process

Process is more valuable than product. Here is why.What is the true value of the world's best companies? Obviously it is not the products they sell since they relinquish them once they produce them in exchange for cash. Rather, it is total ownership over their production process that gives …

How to be 'productive'

“If you want to go fast, go alone. If you want to go far, go together.”African ProverbThe solo developer can appear to cover far more ground than a team until the rest of the team needs to contribute. Only then will it be clear how coupled the developer …

Career Progression in Software Engineering

ProgrammerA programmer knows how to turn user requirements into code. The programmer should have adequate experience but still need to accrue experience in producing high quality code.Software DeveloperA software developer is a programmer who has learned additional professional skills to turn user requirements into code with a …

Declarative vs. Imperative

Most programming languages are designed to be applied imperatively i.e. the programmer has to lay down the sequence of steps to be taken to modify the input to the required output. This lends itself quite naturally to algorithmic thinking. On the flip side, declarative design is built around the idea …

The Key to Great Design

...is good boundaries.

Python Exit Codes

~$ python Python 3.7.4 (v3.7.4:e09359112e, Jul 8 2019, 14:54:52) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.EX_OK 0 >>> dir(os) ['CLD_CONTINUED', 'CLD_DUMPED', 'CLD_EXITED', 'CLD_TRAPPED', 'DirEntry', 'EX_CANTCREAT', 'EX_CONFIG', 'EX_DATAERR', 'EX_IOERR', 'EX_NOHOST', 'EX_NOINPUT', 'EX_NOPERM', 'EX_NOUSER', 'EX_OK', 'EX_OSERR', 'EX_OSFILE', 'EX_PROTOCOL', 'EX_SOFTWARE', 'EX_TEMPFAIL', 'EX_UNAVAILABLE', …

Hardlinks: Fun facts

First we will create a discardable directory calls rm.dir into which we shall add two directories: dir1 and dir2. ~$ mkdir rm.dir && cd rm.dir ~$ mkdir dir1 dir2 Now paste some rubbish into a dir1/file1.txt (find your least favourite website and copy and paste everything):~$ cat > …

Simplicity

The essence of simplicity is economical i.e. simplicity can only be accomplished through economical activities. By ‘economical’ I mean characterised by the academic sense in which we allocate scarce resources that have alternative uses. There are only four such resources that exist: space, time, matter and knowledge. Everything …

What Architects Do

If you are interested in building a house, unless you are an architect, you would do well to consult an experienced architect. You might have a good idea how your future house should look like but you wouldn't know much about how to bring together these various desirable elements into …

Generic Architecture for Interactive Application

There are four parts to any interactive application:core runtimeinput (optional)config (optional)output (optional)The beautiful things is that with this in mind you can now chain applications knowing which parts link to one another.

Digital Service Taxes are like...

Photo by Robert Gourley on Unsplash...taxing the unknown person at the other end of the line. They should apply equally to all kinds of consultants, physicians, experts etc ... anyone who willingly picks up the phone and charges a fee for a response. The term 'digital service' is actually …

On Knowledge

This is an exploratory article I.e. it will take a long time before I understand what I mean. Knowledge must be explored to be understood. Case in point is this article. I don’t know the full extent of the consequences but over time time a clear structure will emerge. The …

Google Cloud Associate Cloud Engineer Exam

While it is still fresh in my mind, I thought I would write my impressions of the Google Cloud Associate Cloud Engineer (ACE) certification exam, which I sat for this morning (Saturday 19th December 2020) online proctored exam. I'm happy I passed though there were moments where I was tempted …

On Productivity

Here is a collection of lessons I have learned and am learning on how to be productive. They are in no order. I’ll refine them over time but here they are in their raw glory to exemplify what I mean. Use realistic deadlines. Deadlines focus the mind. Focus in one …

Intelligence Can...

...only be discerned by intelligence. It is as though it were a handshake of the mind. This means that that which is perceived by intelligent discernment must have been wrought through intelligence and no other means.

One Possible Reason for Incomprehensibility...

...could be that there is nothing to comprehend.

Categories of Study

SpeculativeDescriptiveProductiveAesthetic/CreativeSpeculativePure reason unbound to spatial-temporal world; concerned with ontological analysis i.e. the reasoning on the nature of being for various entities e.g. philosophy is concerned with the substance of reason, natural theology is concerned with the nature of the divine, computing has to do with …

Money and Work

The ideas of money and work are intimately bound, particularly in an economy where the presence, nature and role of money is paramount. Here are some rough thoughts I'm processing about money.What money is very strongly influences the kind of work that gets done. This is because, by its …

TV is...

Tele values - a means by which values can be disseminated to a wide audience for synchronisation. visual conditioning - a means of training your intuition to trust on visual appearance. By its very scarce nature presentation on tv goes through a very strict selection process leading to an extreme …

The Essence of Testing

“Code without tests is broken by design.”Jacob Kaplan-MossI love that quote. It captures it all. Lacking tests is not just bad practice but effectively bad design. Bad design means that something is wrong with the product. A badly designed umbrella is not fit for purpose and could even …

Bad Design...

... is like taking the wrong road. It doesn't matter how fast you travel if you're going the wrong way. It is always cheaper to pause, reorient yourself then proceed than barrel down the wrong path with armfuls of zeal.

Your JD

As a writer of software your primary task is to hide complexity. Think about it: by writing code you employ various abstractions (functions, classes, modules, commands, API etc.) which represent functionality for some user. You would do well if the user never, ever needs to know anything about the …

Multiplicity Admits Combinatorial Complexity

Simplify your counting.There are only three numbers: zero, one and many. You should have no (zero) bugs, one way to do things right and many examples that test for usability. If you have many bugs, multiple ways to do things right and no examples, what …

"Show me your hands!"

This is a great ideal to aim for: that at any time you can be asked to step away from your code and, not only does it continue to function superbly, but those who carry it forward want to reach out and hug you for the fabulous work you've done. …

On Ideas

A good idea at the wrong time is a bad idea.

Hacking Is Like Drafting

All the revision in the world will not save a bad first draft: for the architecture of the thing comes, or fails to come, in the first conception, and revision only affects the detail and ornament, alas!T. E. LawrenceI've previously talked about the three stages of writing software …

From A Physical to Logical Economy

There are two parts to our natural human experience*: the physical and logical. The physical relates to our tangible selves and the logical to our mental selves. Our interactions between these two worlds are so tight that they appear largely seamless, especially given the prevalence of logical tools such as …

Your Secret Weapon Is Your Intuition

Wisdom is the principal thing; therefore get wisdom: and with all thy getting get understanding.Proverbs 4:7 (KJV)One of my teachers fascinated me at his ability to make lightning fast assessments. I was blown away by his snap assessments even without crunching through the details - just a …

Types of Refactor

Name refactor - rename an entity (variable, function/method, class) for clarity in order to reduce logical tensionExtract variable - if a sequence of logic starts repeating itself it's time to create an entity from the snippetExtract method/function Extract class - inheritanceExtract module - when you begin to …

Bugs Make You Strong

Bugs make you strong.Photo by Harish Shivaraman on UnsplashIt sounds counter-intuitive but every bug you find is one step close to making your code base stronger and more reliable. Naturally, we try to avoid any unpredictability in our code almost as if we want to snip all the …

Think Like A Gardener

Great code is grown not built.It is dangerously incorrect to think of code in the same way we think of physical products; that once built they are complete. Software is never complete. The correct metaphor for a programmer is a gardener, who endlessly tends to his garden to bring …

Layers are Good

Photo by Annie Spratt on UnsplashI find that, invariably, code that has been thoroughly pondered over tends to organise itself into layers. For example, for most of the code I write, which runs in the shell, there will be:a presentation layer, which handles commands and options and returning …

Reading: The Part of Writing Great Software

Writing great software is 99% reading documentation.I've previously written about the stages of software development. They are the hack, refactor and integrate stages. The reason the first stack is called 'hack' is because it's usually a representation of a hurriedly put-together product - something you've hacked, …

Your Team: Which Image Best Describes It?

The team of wild horses on the left are not without strength. They are obviously heading in the same direction, which is a good thing. The problem is that they have not been harnessed effectively so that, while they might be producing individually they are not really working as a …

Emergent Design in Software

It is easy to just get going in writing software which is designed for a narrow feature scope. Usually, we can even get away without employing any pen and paper. However, as the feature scope expands it gets harder and harder. Under the waterfall approach the solution was to rigorously …

So, what is computer programming?

A great understanding is transformational. With it your vision is clear and the vista plain. Understanding what computer programming is will amplify your effectiveness at it. In this brief post I will outline the essence of computer programming.A Computer Is A MachineComputers are machines. In the early days …

Stages in Software Development

When producing any software you will ordinarily go through three stages in order to deliver excellent value. These are hack, refactor and integrate. The quality of output will depend whether and how well you perform each of these steps. Poor quality software is marked by most if not …

Entropy in Software

The simplest way to describe entropy is a measure of disorder in a system. A system with a high level of disorder is said to have high entropy. A more software-relevant way to describe entropy is the additional information required to describe a software system. The task of writing software …

Successful Software

Software is successful is useful software. Software is primarily successful if it meets the users' needs. Period. Anything else is bonus material. If a software tool has everything else but lack usefulness then it has failed.There are two type of users: consumption users and production users. Consumption UsersConsumption …

The Dark Side of Object Orientation

Object oriented software design is praised for the fact that it provides an easy mental handle by which a developer may reason about a software entity. However, the bit that is often ignored is the fact that the easy mental handle requires work in order to be teased out. I …

On Roman Numerals

OK. Quick note. I've just come across a Python package for converting Arabic to Roman numerals. Great! But I think it can be improved in several ways.Add documentation even if just in the README file to show how to use it.Add an entry point for users to run …

Keep It Up!

The Google Way to Site Reliability Engineering - https://landing.google.com/sre/

The Value is in the UI

I've just been watching an interesting YouTube video in which a guy solves a sudoku puzzles starting off with very little. It's quite an interesting watch because you're given a front-row seat into observing someone making a discovery of something quite magical. I've always been fascinated with sudoku not least …

Diminishing Returns

Decrease the size of your web images for all browsers using Google's WebP image format that is supported on all major browsers. They report that you can gain savings of roughly a quarter of a PNG file. PNG files are typically tiny therefore to save even more is very attractive. …

The Business of Science: Model Building

I think it is safe to state without any contradiction that science is in the business of validated discovery of new knowledge. This enterprise primarily consists of appending to the volume of what is known (generally referred to as publishing) through a process that is objective for the …

Searchable list of keyboard shortcuts for Mac OS X 10.5+ (IntelliJ IDEA)

Rider's top keyboard shortcutsOpen Solution or Project   Ctrl+Shift+OShow Action List   Alt+EnterSearch Everywhere   Double-ShiftNavigate To…   Ctrl+Shift+NFind Usages   Alt+F7Select In...   Alt+F1Settings...   Cmd+,Find Action...   Cmd+Shift+AGenerate...   Cmd+N Ctrl+EnterBuild Solution   Cmd+F9Debug...   Ctrl+Alt+DView Breakpoints...   Cmd+Shift+F8Attach to Process...   Alt+Shift+F5VCS Operations Popup...   Ctrl+VRefactor This...   Ctrl+TInspect This...   Cmd+Alt+Shift+AFinding everythingSearch Everywhere   Double-ShiftFind...   Cmd+FFind Next / Move to Next …

Automatic Segmentation of Audio by Speaker

ProblemI recently attended a meeting in which I had to write notes from the discussions. I was feeling lazy so I decided that I would record the audio on my phone then make notes from that. That part worked really well until I sat down to make the notes. …

Class attributes

Here's an exploration of some weird Python functionality: how do we intercept class attribute access?The following won't work.class A: x = 1 def __getattribute__(self, name): print('{}.__getattribute__({})'.format(self, name)) return super().__getattribute__(name) print('A.x =', A.x) # A.x = 1 What about setting __getattribute__ to be a classmethod?class A: x …

Using metaclasses to use domain-specific protocols

In Python (3+) it is impossible to create a class which is not a subclass of object. This means that the protocols associated with the class and instances of the class are tightly bound to object. Metaclasses provide a way to extend this so as to have additional …

Interesting and useful links

Emoji Cheat Sheet (view in private tab - might plant tracking cookies) https://www.webfx.com/tools/emoji-cheat-sheet/ Git Documentation https://git-scm.com/

Last will and testament...

Here is an example code snippet that executes 'last rites'. My current use-case is to handle final tasks on a cluster job management system when the script violates some execution criterion e.g. exceeding memory quota. It's helpful if the job could perform some task(s) that indicate that it had to …

Simple Tool to Perform Custom OCR

I'm going to use OpenCV 3.0.0 (https://github.com/Itseez/opencv/archive/3.0.0.zip) though I haven't decided whether I'll use C++ or Python. I might start with Python to build the prototype then move on to C++ for the actual production code. The library documentation can be found at http://docs.opencv.org/3.0-beta/modules/text/doc/ocr.html.In order to …

Africa's False Start

Nearly all African countries began self-rule on the wrong foot. By the time they were 'in charge' of their own destiny they had already developed a healthy appetite for the colonist's produce. Tragic. And no one had the wherewithal to realise the long term implications.Over fifty years on and …

Recipe for Adding Users to Linux

Create the userme@home ~$ sudo adduser <username> I like to use a default password that is easy to remember 'Password01' for first log in.Add the user to any additional groups neededme@home ~$ sudo adduser <username> <group> Enforce change password on first loginme@home ~$ sudo chage -d 0 <username> …

The Recolonisation of Africa

I'm getting a bit concerned about how Africa is being overrun by foreigners to the extent that very little room is being left for the locals to develop their lives. Foreigners are far more competitive and better-placed to take advantage of resources and this presents a difficult situation. On the …

matplotlib Plot Compute Model

I'm not a great fan of matplotlib despite how central it's become in data science. This is because of its API: I find it difficult to learn which suggests to me that it's counterintuitive. As long as I'm working with it I can do a lot but the moment I …

SSH Tunnelling or TCP Forwarding

This has to rank as the most confusing topic in using the Linux command line. It is even confusing using Fugu because there are three user, host and ports that can be used leading to a lot of possibilities to get it wrong. To the best of my understanding, SSH …

What is Object-Oriented Programming (OOP)?

Before I tell you about OOP I have to tell you a few other things that you will understand very easily. Then when I tell you about OOP you will just get it.A programmer is someone who write instructions that direct a computer to perform a desired task. The …

Over-explanation and Teaching

There exists an explanation boundary beyond which an explanation seeks to supplant the listener's responsibility to apprehend the subject matter. It seems to me that most ideas are constructed from a finite set of thought motifs. Our reasoning capacity is designed around these motifs. Therefore, explanations should be made in …

Chama 2.0

Banks are private institutions that determine how national economies develop. They are largely private. The result is that private holdings determine national outcomes. Since banks set interest rates and effectively determine the availability of usable capital they throttle production and prevent the realisation of ideas. At any single point in …

Installing Scipy: You're likely to waste your time if you try using ATLAS

Having grown up on a diet of Ubuntu life has always been so easy. Any time I need to install a library Canonical have done their duty and availed a sanitised version that I can snap on with a quick 'sudo apt-get' or 'sudo aptitude'. However, it's not a bed …

Our present opportunity for self-indoctination

We are experiencing something new in the history of Mankind.At no other time has knowledge been available for the asking as it is today. Just twenty years ago the Internet did not exist, mobile phones were in their early forms and the only access to a self-prepared curriculum was …

Words from the past: Doctor

It struck me some days back that my current pursuits are more suited to my intentions than I earlier thought. See, I'm a teacher at heart. I derive exceeding joy in seeing the lights go on in people's minds. I've been a beneficiary of some of the best teachers on …

The Impediment of Terminology

I find statistics hard. Unnecessarily hard. Is it becomes I'm low on neurons? Hardly! It has more to do with the nature of the discipline and how it has evolved.One of the biggest qualms I've had against both probability and statistics disciplines is the fact that they are mostly …

Working with SSH Public Keys

This post is meant to be more of a sign post than anything else...For any fresh installation, a connection via SSH to a remote machine results in the request to store the public key. You have to respond with the letters 'yes' in order to allow the connection to …

The Forbidden Fruit

"And the LORD God commanded the man, saying, Of every tree of the garden thou mayest freely eat: But of the tree of the knowledge of good and evil, thou shalt not eat of it: for the day that thou eatest thereof thou shalt surely die."- Genesis 2:16,17 (AV) …

Javascript and the DOM

The DOM now makes sense! For the longest time I was completely perplexed as to the rationale behind Javascript. In object-oriented design, the fundamental concept is that of a data structure referred to as a class from which objects ('classes at work') are instantiated. The class is the abstraction - …

Pointers (References) in Programming (especially C/C++)

While learning how to write computer programs in C is quite straightforward for the first few miles, it is not until pointers are conquered that a whole new world opens up. This is not to say that pointers are complicated and hard to learn. Rather, much of the power in …

My Argument for Intelligent Design

How do we test for intelligence? Well, suppose that we make an observation of an ordered repeatable event. Since the event is ordered it is not random and since it is not random there exists a causal pathway that it takes to produce the observed result.We can study this …

Where's the Problem?

I repeatedly encounter this situation. It is a very innocent problem but one that, to me, makes a world of a difference.Some technical books have mastered the art of giving explanations. They can describe any algorithm, process or mathematical model with incredible detail but they forget the most important …

It's OK to Re-Discover

I had a heated debate with a friend yesterday. It centred on something I consider quite sensitive yet immensely pivotal. We had previously touched on it in another argument but yesterday's argument brought things to a head.My argument is that Africa needs to discover things for herself. When I …

Compiling and Linking with GSL

Command-line arguments for GSL compilation. Very important!in reference to: "2.2 Compiling and Linking"- Compiling and Linking - GNU Scientific Library -- Reference Manual

Using random number generators in C from GSL

Very useful information about using the GSL in generating pseudo-random numbers with a uniform distribution.in reference to:"18.4 Sampling from a random number generator"- Sampling from a random number generator - GNU Scientific Library -- Reference Manual

Fallacies of Economic Development and Simple Development Formulae

The one notion that repeatedly comes up in development debates is that the government has a central role in creating both the impetus and oil for the development machinery to function. While superficially sensible, I think it fails miserably for the primary reason that the government cannot be held accountable except, …

Breaking the Rut of Africa's Poverty

If you're plumply tucked against your Korean sofa and expect Africa's liberation to arise from Africa's present leadership then you'll die a cold death. The African Union, while the de facto representation of Africa's political will, cannot be relied upon to bring to her what she needs. It's like high-brow …

Path to the World

Internet is like the butter of computers. If you are running a stand-alone computer you are sorely missing out. But given that we are still awaiting a fiber link at the coast to link us to everyone else at blinding speeds the challenge is to find a reliable and cost-effective …

Ports: at last!

Now, I don't want to seem like a block-head but I'm sure that's what anyone who reads this will think. It has taken the longest time to finally grasp the idea of what ports (in TCP/IP) are. If you are like me you will probably gain; if not, you might …

The Pivotal Role of Modelling in Education

It has just distinctly come to my mind that the education systems we deploy are designed with a built-in weakness.Most disciplines are structured in such a way as to require understanding of several fundamental phenomena. While I was at the University of Nairobi, I took what, at the time …

Where Education Systems Fail: Immediate Relevance

I think understanding is enhanced by starting with examples and I'll do just that.Yesterday, I finally appreciated recursion. For those of you who are not into programming and logical problem solving, recursion is defined as applying a series of steps used in solving a problem (algorithm - applying an …

Passive Learning

Extreme frustration!!! I'm not making any headway in French. i can't put my finger on what I have learned at the end of every class. So, what's the point? There has to be a better way... or no way at all!I think the human mind is amazing in the …

Mind Mapping

The learning process is a sacred process: an opportunity to construct in one's (self or others') a model that will facilitate their interaction with the real world. Irrespective of the application domain, the teacher's task can be more successfully achieved if the right mental map is built - it is …

The Fear of the Unknown

I think the reason we fear the unknown is that we are left at the mercy of 'no information' and so we easily replace the object of our fear with our greatest fear, or we find a way of creating a relationship between our greatest fear and the subject at …

The Learning Process

It has just occurred to me how many times we make the learning process unnecessarily complicated. I recently read an article about Database Design. Wonderful article! In fact, quite a number of concepts clicked in place. However, I had the patience to endure terminology. Quite a bore. I think there …

The Rigors of Learning a New Language

What are the challenges that one will face when learning a new language? Given that I am presently learning (or relearning) French, I have put some thought to the matter and I would like to now put them down on paper (on e-ink).I have identified three main challenges. I …

Principle of Extension

I know that I will need to put this in more precise words but I think it is an interesting idea.Most computational devices run on an interesting principle which I think is the one reason we shall not be able to build Artificially Intelligent Systems that are as intelligent …