When in the Course of human events, it becomes necessary for one people to dissolve the political bands which have connected them with another, and to assume among the powers of the earth, the separate and equal station to which the Laws of Nature and of Nature’s God entitle them, a decent respect to the opinions of mankind requires that they should declare the causes which impel them to the separation.

Services have to be independent. They can achieve Independent Scalability and Independent Lifecycle by having Independent Data, Independent Code and Independent Deployments.

Independent Scalability

The only way to scale monolithic system is to scale it horizontally. Splitting to independent components allows to scale each component individually. This not only saves your resources, but, most importantly, does not allow parts of the system to starve other parts. You may need to think about independent scalability when you have independent operations (APIs) with:

• different resource needs (e.g. CPU- vs. memory-bound).
• different SLAs (fast vs. slow).
• different load (request rate)
• different reliability requirements
• different clients (e.g. end users vs. API clients)

Independent Lifecycle

When you have parts of your system evolving at a different pace, you do not want to tie together their delivery cycles.

Independent lifecycle of services gives us an ability to update any service without impacting the whole system, as long as the change is backwards-compatible. And, with well designed services, we are getting also encapsulation of the risk — failing service should fail only some of system functionality, failing component inside a monolith will crash everything.

Independent Data

We should never depend on internals of data managed by the services. It is their “implementation details”. Service should be always free to change underlying data format and/or storage. Services accessing the same data directly (or even having an easy way to do so) either are already too tightly coupled, or will be tomorrow.

There are only two ways to achieve data independence: to have well-defined bounded context for a service, managed in the independent data store; or to have no data at all (stateless services).

Independent Code

Services should never have code dependencies on each other. Code for each service is its implementation detail, and should be completely hidden from any other service. They can depend on common libraries, but only if these libraries have much higher stability than services themselves. Strictly speaking, one can achieve independent scalability even with code dependencies, but independent lifecycle is impossible if code of one service depends on another’s.

Independent Deployments

Services have to be deployable independently. As soon as you are in a situation where you need simultaneous/orchestrated deployment of multiple services, you are doing something wrong.

Applying Independence Test

With all these independence clauses we can create a rule for when, why and how to make your services independent:

• You should look at splitting your monoliths to services when and if you need (and I mean it, need, not want) Independent Scalability and Independent Lifecycle
• To do this, you have to make your Data, Code and Deployments Independent

This is an updated version of declaration that hat its first appearence in the Can I Haz Name? post

My First Law of Distributed Object Design: Don’t distribute your objects

Everything in our life has its price. Services give a lot of benefits, but you are not getting them for free. One should always very carefully consider the price, and readiness to pay it. There are visible and hidden costs of services, including, but not limited to Latency, Complexity and MPOFs

Latency

Services need to talk to each other. It means that each request will have to be transferred over the network. Messages have to go through series of transformations - from client’s “language”, to network one, and to one that service understands (and the same on the way back, if your request needs a reply). If you have (and I hope you have) security policies applied to communication - add expenses for authentication, potentially - for encryption and decryption too. All these elements add precious milliseconds (tens? hundreds?) to each request processing. Multiply this by amount of services involved - and you can get pretty significant delays. Are you ready?

Complexity

Each time you are adding a new service to your system, you add one more entity to think and care about. Reasoning about architecture is like juggling knives. How many can you do? How many you can add without cutting yourself? What about the case when you have to add one more?

The latter pretty often leads to an opposite extreme - attempts to piggy-back unrelated functionality on existing services (creating new monoliths).

MPOF

This is common understanding that a single point of failure is bad. But multiple points of failures may be even worse, and each of your services becomes such point. First of all, the more points of failure you have, the higher is the probability of failure. And, with SPOF, the state of your system is just “failed”. With MPOFs in case of failure your system is “partially failed” - and you need to understand what it means and how to manage your system’s consistency.

Simple Service Oxymoron

Not once in my practice I’ve asked “Why did you create this service instead of embedding component that would do the same?” and heard “Why not? This service is so simple!”

There ain’t no such thing as a “simple service”. Each service brings in a lot of baggage. You need to manage tests, builds, deployments, logging, monitoring, etc. This is not even counting for what is probably makes your service a service - all the messaging infrastructure with its own dependencies. Somebody will have to maintain it.

And you are paying all the latency, complexity and MPOF taxes.

Conclusion

Just don’t. Only real scalability and/or extensibility bottlenecks should force you to add new services to your system.

Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization’s communication structure.

http://www.melconway.com/Home/Committees_Paper.html

Let look a little closer at what it really means for software organizations, and how communication structure impacts the software system.

Initial design of a software system may have nothing to do with the communication structure. But most (if not all) software systems are evolving in time, and that’s where the Conway’s Law really starts to take its effects. Very similar to the second law of thermodynamics, the evolving software system entropy always increases. And this increase happens in the framework of the communication structure.

When system needs to change, the change can usually happen in multiple places. When person or group decides where to make the change, this decision is influenced by how the information flows inside the organization:

When a change can happen in multiple places, the place closest in communication framework to the change owner (person or group responsible for making the change) will be chosen.

This means that in time changes tend to group around communication “islands” in the organization structure. As there is always some randomness element in determination of who owns the change (being in reality the first design decision, usually made with lack of sufficient information), it results in mix of hits and misses. Hits result in high cohesion of the software solution, misses - in tight coupling.

The wider the communication channel is, the higher the software system cohesion and coupling will become in time.

There is a scientific term for a software system with both high cohesion and coupling: it is called a mess.

Let say we are talking about a good organization, where intra-team communication channels are the widest.

Any evolving software system managed by a single team will eventually become a mess.

Please do not fool yourself that you can avoid this eventual mess by applying some discipline and/or practices. The law is the law!

If this software system is composed of multiple components, this mess is architectural. And we all know that the architectural mess is the most expensive one to fix.

To reduce the cost of the mess created, each team should manage as little software components as possible (preferably - a single component)

But there is another aspect of a team that has its important impact: the size. The more team members we have, the more communication channels are intermingled in a single high-bandwidth communication ball. It means that:

The bigger teams are, the bigger mess they eventually create.

So, is minimizing team (and component) size the way to reduce the eventual mess? Probably - yes, but here is another impact of the communication structure: the smaller the components are, the more of them are impacted by any single change in the system. And communication channels between teams have much lower bandwidth. Leading to:

The smaller teams are, the longer it will take to do a change.

So, small teams are bad. Big teams are bad. And we are again playing the balancing game. The goal of this game is to compare the impacts of intra-component mess and inter-team communication on the delivery speed. Which one slows you more?

Bonus. Quotes from the original Melvin E. Conway’s article

That kind of intellectual activity which creates a whole from its diverse parts may be called the design of a system.

Most design activity requires continually making choices, Many of these choices may be more than design decisions; they may also be personal decisions the designer makes about his own future.

The full-blown design activity cannot proceed until certain preliminary milestones are passed. These include: 1. Understanding of the boundaries, both on the design activity and on the system to be designed, placed by the sponsor and by the world’s realities, 2. Achievement of a preliminary notion of the system’s organization so that design task groups can be meaningfully assigned.

the very act of organizing a design team means that certain design decisions have already been made, explicitly or otherwise.

Once the organization of the design team is chosen, it is possible to delegate activities to the subgroups of the organization. Every time a delegation is made and somebody’s scope of inquiry is narrowed, the class of design alternatives which can be effectively pursued is also narrowed.

Any system of consequence is structured from smaller subsystems which are interconnected. A description of a system, if it is to describe what goes on inside that system, must describe the system’s connections to the outside world, and it must delineate each of the subsystems and how they are interconnected. Dropping down one level, we can say the same for each of the subsystems, viewing it as a system. This reduction in scope can continue until we are down to a system which is simple enough to be understood without further subdivision.

It is a natural temptation of the initial designer – the one whose preliminary design concepts influence the organization of the design effort – to delegate tasks when the apparent complexity of the system approaches his limits of comprehension. This is the turning point in the course of the design. Either he struggles to reduce the system to comprehensibility and wins, or else he loses control of it. The outcome is almost predictable if there is schedule pressure and a budget to be managed.

As long as the manager’s prestige and power are tied to the size of his budget, he will be motivated to expand his organization. This is an inappropriate motive in the management of a system design activity.

Probably the greatest single common factor behind many poorly designed systems now in existence has been the availability of a design organization in need of work.

To the extent that organizational protocol restricts communication along lines of command, the communication structure of an organization will resemble its administrative structure. This is one reason why military-style organizations design systems which look like their organization charts.

organizations which design systems (in the broad sense used here) are constrained to produce designs which are copies of the communication structures of these organizations.

a design effort should be organized according to the need for communication.

Because the design which occurs first is almost never the best possible, the prevailing system concept may need to change. Therefore, flexibility of organization is important to effective design.

I guess it is easier to use a new name (Microservices) rather than say that this is what SOA actually meant

I guess it may be even easier to say what SOA actually meant. Or, at least - what I believe it had in its heart. In this post I am going to try to see how can we define SOA the way that:

• captures its spirit
• does not have elements that damaged the whole idea (WS-*, ESB, etc.)
• goes even further and does not include anything that overly-limit SOA application

In essence, we are looking for a lean SOA definition (to make sure, it is not “lean SOA”, but “lean definition”).

Here is the SOA definition from one of my favorite books on the subject, SOA Patterns by (the same) Arnon Rotem-Gal-Oz:

Service-oriented architecture (SOA) is an architectural style for building systems based on interactions of loosely coupled coarse-grained, and autonomous components called services. Each service exposes processes and behavior through contracts, which are composed of messages at discoverable addresses called endpoints. A service’s behavior is governed by policies that are external to the service itself. The contracts and messages are used by external components called service consumers.

Pretty close to this is description from “SOA in the Real World”

With a Service Oriented Architecture the application’s functionality is exposed through a collection of services. These services are independent and encapsulate both the business logic and its associated data. The services are interconnected via messages with a schema defining their format; a contract defining their interchanges and a policy defining how they should be exchanged.

Let try to be pragmatic - these definitions already got rid of all “vendory” and “enterprisy” SOA rubbish; but are there any parts that are still optional? And is there anything missing? We want our definition to be flexible enough, to be able to capture all styles that still can be categorized as an SOA.

Key components of these definitions:

• Service - loosely coupled, coarse-grained, autonomous, independent components in the first defenition; independent and encapsulate business logic and data in the second. Autonomity and encapsulation of logic and data map very well to the bounded context. Coause-grained - seems to be directly size-prescriptive, we do not want to fall into the same trap as microservices did.

  It also seems that one characteristic is missing. There is nothing about what makes services so flexible and powerful - ability to horizontally scale them.

• Contract - scoped differently in these definitions. In the first one it captures messages and endpoints, in the second - definition of interchanges. There is one aspect that in both cases is not included explicitly in the definition, but described outside of it - both assume contract being a software artifact. Having experience with different styles and incarnations of SOA, I’d characterize explicit artifact-based contract as a good, but optional practice. Without artifact - clients should know how to call service and what to expect in return.

• Endpoint (only in the first definition, component of a contract). Where to call a service. I’d move it out of definition too - particular frameworks may hide notion of an endpoint from clients completely. Again, clients should be able to reach services - common sense.

• Message - both definitions assume that communication happens with the help of messages. Arnon explicitly distinguishes them from RPC. I’ve seen in the past architectures that managed to use RPC without sacrificing any other important qualities.. As we want to give as much freedom as possible in our definition, let drop it too. But we cannot drop them completely, as they cover one of the most important aspects - communication between services. Without communication services are just … services. Defined communication makes them architecture.

• Policy - this is a non-functional element of a contract. It can capture aspects like availability, latency, security, etc. The same as with the contract, requiring policies as a software artifact unnecessarily limits application of SOA; without artifact - they are good practices and common sense elements.

If we remove everything that is not required, and add missing element, we can see our new lean definition of [what] SOA [actually meant]:

SOA is an architecture defined in terms of services and their communications. Services are independent, loosely coupled and horizontally scalable software components organized around bounded contexts.

The Technical Debt Trap by Doc Norton

Technical Debt has become a catch-all phrase for any code that needs to be re-worked. Much like Refactoring has become a catch-all phrase for any activity that involves changing code. These fundamental misunderstandings and comfortable yet mis-applied metaphors have resulted in a plethora of poor decisions. What is technical debt? What is not technical debt? Why should we care? What is the cost of misunderstanding? What do we do about it? Doc discusses the origins of the metaphor, what it means today, and how we properly identify and manage technical debt.

Software Architecture vs Code by Simon Brown

Software architecture and coding are often seen as mutually exclusive disciplines, despite us referring to higher level abstractions when we talk about our software. You’ve probably heard others on your team talking about components, services and layers rather than objects when they’re having discussions. Take a look at the codebase though. Can you clearly see these abstractions or does the code reflect some other structure? If so, why is there no clear mapping between the architecture and the code? Why do those architecture diagrams that you have on the wall say one thing whereas your code says another? In fact, why is it so hard to automatically generate a decent architecture diagram from an existing codebase? Join us to explore this topic further.

One Hacker Way by Erik Meijer

One Hacker Way, a Rational Alternative to Agile

Presented at Reaktor Dev Day 2014

reaktor.fi/blog/erik-meijer-software-eating-world/

Nanoservice is an antipattern where a service is too fine-grained. A nanoservice is a service whose overhead (communications, maintenance, and so on) outweighs its utility.

—Arnon Rotem-Gal-Oz

So I decided to look for a better name. It should be sticky and easy to remember. It should not focus on size, or any other questionable aspect. And it may be fine if it still means nothing - as long as this nothing does not have confusing semantics.

Let start with abbreviation. It probably will end with SA - Service Architecture. Let leave RSA to Reactive Manifesto camp. NSA - too bad association. BSA - should be reserved for BS Architecture.

I think I like USA. Let create Ultra Services Architecture!

And ultra is a good prefix for #ultraservices. It can mean anything you want! Ultra-scalable, Ultra-fast. Ultra-happy. And, if you feel very adventurous, you can still try to do ultra-small (just let me know upfront before I try to use your app)

And, as we are talking about USA, it needs

Declaration of Independence

When in the Course of human events, it becomes necessary for one people to dissolve the political bands which have connected them with another, and to assume among the powers of the earth, the separate and equal station to which the Laws of Nature and of Nature’s God entitle them, a decent respect to the opinions of mankind requires that they should declare the causes which impel them to the separation.

[Ultra] Services have to be independent. Their independence is governed by three main aspects:

Independent Scalability

The only way to scale monolithic system is to scale it horizontally. Splitting to independent components allows to scale each component individually. This not only saves your resources, but, most importantly, does not allow parts of the system to starve other parts. You may need to think about independent scalability when you have independent operations (APIs) with:

• different resource needs (e.g. CPU- vs. memory-bound).
• different SLAs (fast vs. slow).
• different load (request rate)
• different reliability requirements
• different clients (e.g. end users vs. API clients)

Independent Lifecycle

When you have parts of your system evolving at a different pace, you do not want to tie together their delivery cycles. Independent deployment of services gives us an ability to update any service without impacting the whole system, as long as the change is backwards-compatible. And, with well designed services, we are getting also encapsulation of the risk - failing service should fail only some of system functionality, failing component inside a monolith will crash everything.

Independent Data

We should never depend on internals of data managed by the services. It is their “implementation details”. Service should be always free to change underlying data format and/or storage. Services accessing the same data directly (or even having an easy way to do so) either are already too tightly coupled, or will be tomorrow.

There are only two ways to achieve data independence: to have well-defined bounded context for a service, managed in the independent data store; or to have no data at all (stateless services).

Applying Independence Test

With three independence clauses we can create a rule for when it may be a good idea to look at Ultra Services Architecture

You should look at splitting your monoliths to services if:

• you need (and I mean it, need, not want) Independent Scalability and Independent Lifecycle
• you can make your Data Independent

And making your data independent (by defining a bounded context and moving it to a separate store) should always be the first step.

BTW, please do not be fooled by the semi-jocking tone of the USA part of the post. I’ve started to work on these independence principles way before microservices fad have started, so there was no need for a name.

TL;DR: Name them however you want (though probably not microservices), but always follow three rules of service independence - Independent Scalability, Independent Lifecycle and Independent Data

You can see an updated version of this declaration at Services: Declaration of Independence

In March of 2014 James Lewis and Martin Fowler published an article in an attempt to summarize the meaning of this term (and contributing to the hype wide spread).

Interlude: Conspiracy Theory

The term “microservice” was discussed at a workshop of software architects near Venice in May, 2011 to describe what the participants saw as a common architectural style that many of them had been recently exploring. In May 2012, the same group decided on “microservices” as the most appropriate name.

— James Lewis & Martin Fowler, March 25, 2014

Yep, anonymous workshop of anonymous architects anonymously decided on “microservices” as the most appropriate name… BTW, were they all from the same anonymous company?

Originally (and, AFAIK, even in the first revision of the Lewis & Fowler article) Microservices had a strong emphasis on Micro, and even referred to a 10-100 LOC size guideline. After getting pretty strong pushback on any size limitations from many practitioners, all the size references were gone (finally a reality check?), and “Micro” part started to be more and more frequently referred as being “just a name”

With “Micro” semantics dropped out of the picture, it became obvious (to many people, though there is no real consensus here neither) that Microservices are just SOA striving for a new identity, because an old one had too bad reputation.

I guess it is easier to use a new name (Microservices) rather than say that this is what SOA actually meant - re http://t.co/gvhxDfDWLG

— Arnon Rotem-Gal-Oz (@arnonrgo) March 16, 2014

@martinfowler @boicy but these are the very principles of SOA before vendors does pushed the hub in the middle, i.e. ESB

— Clemens Vasters (@clemensv) March 16, 2014

@UncleGene Micro-Services are just SOA done properly – simple functionality, well-defined interfaces, minimal dependencies

— Randy Shoup (@randyshoup) November 7, 2014

Microservices is SOA, for those who know what SOA is.

— Steve Jones March 18, 2014

So, to understand what “Microservices” really are, we should first understand what is SOA (and what is SOA done properly). Rose may smell sweet by any name, but if you step into [something] - you can name it a rose (or micro[something]), but smell will not change until you clean up your shoes.

You can see my “tongue-in-cheek” definition here, please stay tuned for the real one coming soon.

But for now, just one takeaway: “Micro” in “Microservices” means absolutely nothing.

Our usual suspect, wikipedia, says:

Service-oriented architecture (SOA) is a software design and software architecture design pattern based on distinct pieces of software providing application functionality as services to other applications. This is known as service-orientation. It is independent of any vendor, product or technology.[1] – http://en.wikipedia.org/wiki/Service-oriented_architecture, 09/18/2014

Let try to digest.

architecture […] is a software design and software architecture design pattern

Awesome! After all these years of debates about differences between architecture and design, in one phrase we can see that they are not only the same, but, at least in the case of SOA, also the same as design pattern!

pieces of software providing application functionality as services to other applications

OK. Got it. Services are providing functionality as services. And they are pieces of software. And, looking at definition, they are not necessarily applications, but have to provide application functionality, and have to be used by applications. Basically, “application of service applications as services to application services”

This is known as service-orientation.

What “this”? SOA?

The only clear and unambiguous part is “It is independent of any vendor, product or technology”, so we’ll set it aside for the future reuse.

Curiously enough, the reference there points to the first chapter of “SOA in the Real World” book by some anonymous author, that gives absolutely different, and much more vague definition:

[SOA is] A loosely-coupled architecture designed to meet the business needs of the organization.

Whoa, whoa, is it assuming that there can be an architecture that is designed not to meet business needs? Quite possible, but as we do not want get fired, let skip this part:

SOA is a loosely-coupled architecture.

Nice. Does everybody understand now what are we going to talk about?

Let look for other definitions. Here is the next one, and as it is compound definition, let consume it piece-by-piece.

Service-Oriented Architecture (SOA) is an architectural style that supports service-orientation.

I absolutely and completely agree. I can continue the line: Object-Oriented Design (OOD) is a design style that supports object-orientation. Test-Driven Development (TDD) is a development style that is driven by tests. Central Intelligence Agency is an intelligence agency that is central (or an agency whose intelligence is central? Looks like this one may benefit from such definition).

Let continue:

Service-orientation is a way of thinking in terms of services and service-based development and the outcomes of services.

No comments. Though to make it better, I’d rather mention services couple more times. Going forward.

A service: - Is a logical representation of a repeatable business activity that has a specified outcome (e.g., check customer - credit, provide weather data, consolidate drilling reports) - Is self-contained - May be composed of other services - Is a “black box” to consumers of the service

When I first started to read this, I’ve stumbled on the first point’s “business activity”. Not being sure what is really meant here, I looked couple paragraphs below, and found some clarification. Here is a statement about “SOA architectural style” (yes, defined separately):

It is based on the design of the services – which mirror real-world business activities – comprising the enterprise (or inter-enterprise) business processes.

Seriously? Services are required to mirror what exists in the “real world”? Let replay this statement in other direction: “Hello, operator? Could you please help me with transcoding my video file? Here are my bits: 1,0,0,1,0….”

Honestly, I have no clue what “self-contained” and “may be composed of other services” mean, especially - in combination. “Black box” is fine, we’ll add it to our collection.

BTW,

This definition of SOA was produced by the SOA Definition team of The Open Group SOA Working Group.

This phrase itself, IMHO, deserves pretty long meditation. Even parsing and tokenization are not trivial.

Let combining all pieces (that we decided to keep) together. SOA is:

A loosely-coupled architecture created with black boxes, independent of any vendor, product or technology

This is my talk at Rocky Mountain Ruby 2013. Slides are available here.

Rails is a framework well known for ease of development. This ease is achieved by a lot of ‘magic’ that happens behind the scenes. One of pitfalls of such magic is a false sense of safety it gives, including sense of safety from concurrency issues for single-threaded environments. You may never discover any problems before the launch, or even after, while your site traffic is pretty sparse. But here comes a glorious moment of popularity - and together with more traffic it brings more and more concurrency-related problems.

In this talk we will look at different aspects of concurrency, from simple ones that are even mentioned in Rails documentation, to more complex problems that even seasoned developers tend to miss or fail to pay sufficient attention to.

• Deconstructing the Framework by Gary Bernhardt
• Agile is the new black by Fred George
• Confident Code by Avdi Grimm
• Delicious Controversy: Docs & Tests by Thomas Meeks
• Boundaries by Gary Bernhardt

Deconstructing the Framework by Gary Bernhardt

Rails gives us M, V, C, routes, and helpers. Some people add observers and concerns, among others. We’ve standardized on presenters. Service objects are gaining popularity. Uncle Bob wants you to add interactors, request models, response models, and entities.

That’s a lot of stuff! Let’s step back: ideally, how do all of these things fit together? Does it make sense to have so many different components? How do different web frameworks project these onto actual components? Most importantly: how does this explain the tangled mess in Rails controllers and how we might fix it?

Agile is the new black by Fred George

“This mid­ 80’s declaration from the fashion industry has become synonymous with radical shifts in the norm of any field. Agile provided such a radical shift for traditional waterfall processes. Yet as Agile has matured, it is redefining itself at a pace that rivals the whims of the fashion industry.

This presentation presents not only the (somewhat obvious) shifts from waterfall to Agile, but the second and third generation of shifts within the Agile community itself. Basics such as automated unit tests are falling away (“Deployment is the new unit test”).

The overall message is to continue to question practices, and strive to understand the reasons behind a practice so that you know when it is safe to discard.”

Confident Code by Avdi Grimm

Are your methods timid? Do they constantly second-guess themselves, checking for nil values, errors, and unexpected input? Even the cleanest Ruby codebases can become littered over time with nil checks, error handling, and other interruptions which steal attention away from the essential purpose of the code. This talk will discuss strategies for writing your Ruby classes and methods in a confident, straightforward style; without sacrificing functionality or robustness. In the process, we’ll cover concepts and techniques points including: The narrative style of method construction The four parts of a method Three strategies for dealing with uncertain input Massaging input with coercion and the Decorator pattern Lightweight preconditions Exterminating nils from your code The chaining and iterative styles of method construction Eliminating conditionals with the Special Case and Null Object patterns Isolating errors with the Bouncer and Checked Method patterns

Delicious Controversy: Docs & Tests by Thomas Meeks

Self-documenting code is a pipe dream. TDD (or BDD) is not the panacea of testing. In the pursuit of test coverage we’ve forgotten what really matters: getting things done. Lets talk about putting documentation and testing into their proper place. Tools that ease maintenance, help other developers join a project, and reduce bugs.

I’m going to go over lessons learned in writing, maintaining, and introducing new developers to 20,000 lines of code. Specifically, how we are testing, documenting, and refactoring our code to stay sane, make the team happier, and get more done.

Boundaries by Gary Bernhardt

Some people test in isolation, mocking everything except the class under test. We’ll start with that idea, quickly examine the drawbacks, and ask how we might fix them without losing the benefits. This will send us on a trip through behavior vs. data, mutation vs. immutability, interface vs. data dependencies, how data shape affords parallelism, and what a system optimizing each of these for natural isolation might look like.

1

options[:days], partial_days = options[:days].divmod(1)

This sparkled my interest: what is divmod, especially - its pretty harsh form divmod(1)? Quick search through documentation - and it starts to make perfect sense: Returns an array containing the quotient and modulus obtained by dividing num by numeric; especially if you instantly remember what is quotient and modulus. Looks like example table in documentation is designed to confuse (including expected output of multiple methods), but for the special case of a positive float.divmod(1) there is much simpler explanation: it just splits a positive floating point number to its integer and fractional parts (+/- error).

Or, in other “words” (this one is true for negative numbers too):

1

float.divmod(1) == [float.floor, float - float.floor]

Examples:

1
2
3
4
5
6
7
8

> 1.5.divmod(1)
=> [1, 0.5]
> 1.4.divmod(1)
=> [1, 0.3999999999999999]
> -5.5.divmod(1)
=> [-6, 0.5]
> -5.3.divmod(1)
=> [-6, 0.7000000000000002]

Let look at other surprising Numeric methods (ones that I either never saw or successfully forgot about).

#abs2

Returns square of self. Nothing to say, other than interesting choice of name. I would never guess what it means.

#coerce(numeric)

Returns an array of a target and an argument if they are of the same type, converts both to Float if they have different types. It would be interesting to see a practical (not in the core code) use of this.

#conj, #conjugate

Returns self for Numeric. Probably - no use for non-Complex numbers

#numerator, #denominator

I am excited about this pair, and really want to find a practical use for them. Documentation is extremely clear: it states that denominator returns denominator, and numerator - surprise! - numerator. Makes perfect sense for Rational, but sounds weird for all other types. In reality these methods return numerator and denominator of a Numeric converted to a Rational

Taking out of the context - try to make sense out of these results:

1
2
3
4

> 1.2345.denominator
=> 4503599627370496
> 1.2345.numerator
=> 5559693739988877

Together - much better

1
2
3
4
5
6

> a = 1.2345
=> 1.2345
> a.to_r
=> (5559693739988877/4503599627370496)
> a.numerator.fdiv(a.denominator) == a
=> true

I could find only one “valid” use for #denominator for non-rational numbers - simplify (is it really simplification?) the check if any number is “practically an integer”

1
2
3
4
5
6

> a = 123.0
=> 123.0
> a.to_i.to_f == a
=> true
> a.denominator == 1
=> true

For #numerator I was able to find only pretty bad usage (not sure who started it as a pattern):

1
2
3

def days_until(date)
  (date - Date.today).numerator
end

Problems here:

• for a non-manipulated Date input, it is no better than #.to_i
• for calculated dates - it is much worse than #to_i as it will purely lie to you.

1
2

> days_until(Date.today + 1.3)
=> 13

#fdiv

Float division. Not sure yet, but I think I like it better than its practical equivalents a / b.to_f or a.to_f / b - seems more explicit.

#nonzero?

Thank you, Ruby, for consistency! All other xxx? methods return true or false. But not this one. Returns self where you’d expect true, and nil for false. Usage example in documentation

1
2
3

a = %w( z Bb bB bb BB a aA Aa AA A )
b = a.sort {|a,b| (a.downcase <=> b.downcase).nonzero? || a <=> b }
b   #=> ["A", "a", "AA", "Aa", "aA", "BB", "Bb", "bB", "bb", "z"]

does not give any clarity for why this inconsistency is necessary. This metthod would make much more sense to me without “?””

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

ActiveRecord::Schema.define do
  drop_table :records if table_exists? :records
  create_table :records do |t|
    t.timestamps
  end
end

class Record < ActiveRecord::Base
end

class CuriousFirstTest < MiniTest::Unit::TestCase
  def test_first_last
    r = Record.create
    Record.create
    r.touch
    assert_equal 2, Record.count
    assert Record.first != Record.last, 'The first of 2 records is also the last!'
  end
end

What are we doing here? We create two records, touching first (this updates the record timestamps), and making sure that the first record is not the same as the last one. Perfectly valid assumption, that almost works.

The test succeeds on SQLite and MySQL (latter - with various engines), but fails on PostgreSQL.

Is it a database fault? Not really. Rails (before 4.0) used a strange asymmetric approach: #last (when statement do not have any explicit order) adds ordering by primary key, but #first does not add anything, thus using default database ordering. No database actually guarantees what is implicit order, but there is still some consistency: at least in simple cases, MySQL always maintains primary key order, but PostgreSQL places most recently modified records at the end.

Bug? I’d say yes, but… This is not the bug that is going to be fixed. Apparently there are developers “in the wild” that managed to rely on this behavior. Thus – this behavior was effectively declared “a feature, not a bug”…

Listening to the teacher’s presentation, I was surprised by how process-oriented the education is (not having US education experience myself). Here is one of the processes that were presented by the teacher as a part of 5th grade curriculum:

Design Process

1. State the Problem
2. Gather Information
3. Explore Ideas
4. Select a Solution
5. Test and Evaluate
6. Redesign

And this is taught in elementary school! One question that I have now is: did most of software engineers skip the 5th grade? Or this is something new that was added to curriculum less than twenty-something years ago? The “Design Process” I see again and again, not only with interns or junior engineers, but also with some experienced people:

1. Pick the Task (or Get Assignment)
2. Pick a Solution
3. Implement
4. Try if it Works (sometimes)
5. Implement Whitebox Tests (only if forced to do testing. In a lot of cases tests are matching implementation one-to-one; if code has bugs – tests are ensuring that bugs are there)

Sometimes it even extends to:

6. Restlessly Defend Picked Solution as the Best One

Nope. Please, go back to elementaruy school. They can teach you how to do your job!

All right, all right, I agree, we do not have time for this, and, more important, our school district is not funded enough to bring you back to school. But I hope you did not skip classes when they tought reading. Or did you? If not, please read what these steps mean (or, at least, how I want you to understand them):

Design Process for Engineers

1. State the Problem

Describe in your words what is the problem you are trying to address. If you cannot clearly describe the problem – you do not understand it. If your friend does not understand your description - you do not understand it.

2. Gather Information

Starting with the most important one – are you really stating the right problem? Or even: did the person that originally stated it understand what she wanted? Is there already a solution for either directly the problem or something similar? How well does it fit your case?

3. Explore Ideas

There is a very high probablility that the first idea that comes to your mind is the wrong one. And do not forget to continue exploring on all steps. And please, PLEASE, do not limit yourself to your ideas only. There are a lot of smart people around.

4. Select a Solution

Remember: there is no selection if you have only one solution.

5. Test and Evaluate

Test not what you did, but whether the problem was solved. Evaluate all steps that were done before – starting from restating the problem. Quite possible that with all new information you have now your understanding of the problem may change.

And one of the most important evaluation aspects – decision whether you have to go to the step #6 or not. Understanding of what is “good enough” comes with experience, so do not trust yourself if you lack one.

6. Redesign

Return to #1. Always. Yes, I know, you think you are ready for #4. Forget what you see at this moment as a “better solution” – you are wrong (even if you are right!).

Bonus #1: Investigation Process (Elementary School Edition)

1. Question
2. Prediction
3. Materials
4. Procedure
5. Data/Results
6. Conclusion

Bonus #2: Writing Process (Elemenatry School Edition)

1. Prewrite
2. Rough Draft
3. Revise
4. Edit
5. Publish

1
2
3
4
5

launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist
/usr/local/opt/mysql/bin/mysqld_safe --skip-grant-tables --skip-networking &
mysql -e "UPDATE mysql.user SET Password=PASSWORD('new password') WHERE User='root'; FLUSH PRIVILEGES;"
/usr/local/bin/mysql.server stop
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist

Different versions may have different paths to be used with launchctl load/unload

1
2
3
4
5
6
7
8
9

s = " To\tbe  or\nnot\tto  be  "
s.split(/ /)
#=> ["", "To\tbe", "", "or\nnot\tto", "", "be"]
s.split(/( )/)
#=> ["", " ", "To\tbe", " ", "", " ", "or\nnot\tto", " ", "", " ", "be", " ", "", " "]
> s.split /\s+/
#=> ["", "To", "be", "or", "not", "to", "be"]
s.split(' ')
#=> ["To", "be", "or", "not", "to", "be"]

From the ruby docs:

If pattern is a single space, str is split on whitespace, with leading whitespace and runs of contiguous whitespace characters ignored.

Note that split by regex without groups omits trailing matches. Undocumented?