Dealing With Ambiguity

Posted on May 24, 2023 by adriankearns

Introduction

“Architect” is a thought-leadership role, and being a thought-leader means dealing with ambiguous problems. You may need to be the solver of ambiguous problems, or guide your colleagues so they can do so.

Dealing with ambiguity is an important skill, otherwise it will slow you down or lead to poor decisions.

Beware of Over-Baking

A symptom of being paralyzed by ambiguity is to over-bake a piece of work – probably one that is comfortable.

Our brains love avoiding difficult or unpleasant tasks in flavor of easier and more enjoyable ones, so it’s important to recognize when you are faced with ambiguity before too much time is lost to avoidance. If you find yourself putting a lot of effort into something fun but relatively trivial – that might be a sign to move on.

For example, imagine some two part process: step one is to conduct some analysis, but you’re unsure what step two looks like. If you find yourself (or someone else) over-baking the analysis (“Oooh, I really need to add some more colour to these charts”) it might be due the next step is not well defined i.e. you don’t know how to start it or you don’t have a plan.

Mitigation Strategies

The following approaches can be useful for dealing with ambiguous problems. These approaches aren’t mutually exclusive – you can mix and match.

Go back to the Objective

One of the first things to do is make sure you haven’t forgotten the objective.

Often we immerse ourselves in a complex problem and get so caught-up in its intricacies that we forget what we’re actually there to achieve. Once we lose sight of our objective we tend to get lost. Therefore, remember to keep coming back to the objective: make sure you clearly understand it and don’t lose sight of it.

Also, you can use the objective as a point of reference – to work backwards from, and that can help penetrate the ambiguity.

Go Back to First Principles

Similar to the previous strategy, a good way to take control of a problems intricacies is to go back to first principles.

“First principle” is essentially a fundamental truth that stands alone. Aristotle defines it as “the first basis from which a thing is known.” In the context of this strategy, first principles refers to things which you know to be true – not things you assume.

Often our understanding of complex problems is a web of linked assumptions; if the overall hypothesis is flawed we can end up getting lost due to circular reasoning – essentially we get defeated by the complexity. Going back to first principles is essentially to throw all your assumptions out the window and start again.

Using first principles helps you work methodically and focus on specific things that matter; “divide & conquer” can be a good complimentary approach.

Give Yourself Time / Space to Think

Complex / multi-dimensional problems will require more mental processing than usual.

The number of variables & possibilities present, plus the variety of disparate information available, will be larger than usual. Therefore, its important to give yourself time to think.

“Chipping away” is a great companion strategy – digest part of the problem and mull it over or sleep on it, then take another bite. As you build up a picture of the problem you’ll develop a mental model that you can fit new things into. Also be ready to change and restructure your mental model as needed – try not to be too rigid.

Just Attack the Problem

Brute force. Like starting a cold engine, you might need a surge of energy to make an initial break-through.

One way to break writers block is to just start writing – even if the initial output is total garbage, the act of forced writing can help kick-start the brain into the right kind of processing, and productive writing will follow. The same can hold for problem solving – sometimes you just need to get stuck in and warm your brain up.

Divide & Conquer

Break a big problem into smaller ones. Concentrate on one. Make progress or recognize when to give it time and come back to it.

Slightly related – rather than trying to solve the entire problem first, see if you can put some “stakes in the ground”, i.e. make concrete progress on some key points and use those as a basis for further work.

Find Themes / Consolidate

Group many moving parts into themes to reduce overall “noise”.
Look for patterns.
See the big picture.

Chip Away

“Death by a thousand cuts” but in reverse.

Putting a small amount of effort into something huge usually seems futile,
but many small efforts added together
will make an impact.

Works well with “giving yourself time/space to think”. Take a small bite at the problem and mull it over.
Supports a sustainable pace. (Its a marathon, not a sprint).
Start early. Expect course corrections.

Defer

Putting-off something difficult by procrastinating is not a recipe for success. That said, problems sometimes have a timing dimension to them, and so picking the right time to tackle the problem is sensible.

The trick is to successfully identify the factors, and recognize that its real and not procrastination.

For example, the work of you and your team will often have dependencies on other projects, teams, decisions and future state strategies; sometimes the direction of these will be unclear. In such cases you may need to get clarity on where they are likely to land before it is worth investing significant effort in related problems.

Share Your Problem

If you’ve been struggling on your own, share your problem with a colleague or a friend. Often an outside perspective is all we need to get us started.

Time-boxing

Not so much a strategy of it’s own but something which applies to many of the strategies above.

Think about how much time to invest in a given approach. If you still haven’t made progress after an appropriate amount of time, ask yourself:

Is this a signal you need to switch to a different approach?
Should we put the problem to the side for the moment and come back to it later?
Is the problem insoluble, and we may need a change in overall strategy?

More Info

This expands on part of the “Consulting Architect – How To Get Sh#t Done” talk I gave at the Geekle Worldwide Software Architecture Summit, May 2023:

Deck, extended response to some audience questions, etc: https://morphological.wordpress.com/2023/05/16/consulting-architect/
recording of the event stream: Worldwide Software Architecture Summit’23 – Career Track – YouTube
Event Schedule & speakers: Agenda- Worldwide Software Architecture Summit 23 (geekle.us)

Consulting Architect

Posted on May 16, 2023 by adriankearns

As (intended to be) presented at the Geekle Worldwide Software Architecture Summit 2023
May 16, 2023 at 9:00 am UTC – May 17, 2023 at 7:00 pm UTC

consulting-architect-geekle-wsas23-may-2023-v0.1 Download

In this talk is based on my lived experience, working as a consulting solution architect.

There’s a skills overlap between consultancy and architecture; and I think architects have an opportunity to be more effective by applying some of the consultants mindset and techniques. In short – “consider being a “the consulting architect”.

The deck includes extensive notes on selected slides, and I’ll be publishing blog posts on selected topics in due course.

One of my motivations behind this talk is that there’s lots of ideas, skills and techniques which (as far as I am aware) aren’t usually discussed in architecture circles, so this is a chance to explore some of those and see how they can help us be more effective architects.

Update – Q&A Session Responses

As much as I like the challenge of answering questions on the fly (in Q&A sessions), because I’m (almost certainly) dyslexic I often don’t have enough processing time to give really good answers. Last night I got three questions, and I’ve done some more processing 🙂

To those who asked the questions – I hope you see this and that its helpful.

How to select / weigh which is hardest easiest problem to start with?

Usually the size of a problem is obvious. Whilst I think it is wise to be mindful about which problems we start with, it’s also something which is easy to over-think.

In my talk a mentioned how the brain has two modes of reasoning: heuristic (intuitive) and analytical; the former is usually pretty good at sizing up the relative size / difficulty of a problem, and that’s often sufficient. Is the relative difficultly immediately obvious? If you have to stop and think about it for too long that probably indicates it’s not easy.

If you do want to be more analytical, below are some of the things I think about when doing problem triage, in roughly the order you may want to consider them. Remember:

You don’t need to apply all of them! Just use whatever works for you to sort the problems.
Triage is just an initial assessment – it’s okay to change your mind later (as to if it’s an easy or hard problem) because all you’re doing is sorting out which problems you want to tackle first.
Because its an initial assessment, use simple coarse-grained measures; for example easy or hard; or easy, medium, hard, unknown. The point is to balance effort and accuracy – whilst we’d like an accurate measure, but it’s probably not worth investing a lot of effort.
A lot of these techniques can be used with different levels of effort: a 30 second thought exercise in your head; a 5 minute doodle on the back of an envelop; a 10 minute whiteboard chat with a colleague.

Things to think about:

Is the relative difficultly immediately obvious? If you have to stop and think about it for too long that probably indicates it’s not easy.
How many unknowns? Easy problems tend to be well understood because there’s not much to them.
Can we change our minds? Architect is about making sound decisions, where we know that once we make those decisions it will be very challenging to change them, so we need to be careful. Easy problems can sometimes also be relatively straightforward to change later.
Blast radius. If you misjudge the problem and it explodes – what’s the likely blast radius? Would it affect just you, the project or the whole enterprise? The bigger the perceived blast radius the harder the problem might be. Also see “risk profiling” (below) which expands on this idea but is more complex.
Is the problem rooted in people or technology? Technology problems tend to sit there waiting for you to research the answer online (easy). Problems that involve other people can be harder because people tend to have their own ideas about things (how dare they!), so you may need to engage with them: take time to understand their perspectives, motivations and priorities, factor any new ideas and information they provide into your mental model; sell them your ideas, influence and negotiate.
“Firsts”. In solution architecture you’ll often be faced with “new” problems, coming up with new patterns, using new technology and so on. This relates to “unknowns” but the nuance here is about doing new things for the first time – you might have a good idea of what needs to happen but because it’s the first time in this context there’s a degree of unpredictability.
Risk profiling: think about the risks that might relate to the problem – hard problems tend to have more risks. For the risks, think about them in terms of likelihood and impact – the more likely they are, and/or the bigger impact they have the more serious you need to take the risk. Harder problems tend to attract more serious risks i.e. more likely and bigger impact. For an initial problem triage you don’t need to think of all the risks associated with it – you only need to identify if it has any that cross a red-line in your head. As soon as it crosses that line put it in the hard basket and move on.

Must read book for developer wanting to migrate to solution architecture?

The developer to architect journey is one I took myself, so it is important to me. I’ve been focused on this topic for a while now and have these resources which may help. They are listed in a suggested reading order.

How do you manage continuous delivery?

This is a very open-end question, but here’s some thoughts:

Is it even your problem? If you are new to architecture, and in your last job you were responsible for delivery (maybe you were a tech lead), then it’s possible the answer is “no”. In such cases you’re mentally still trying to do your old job; you need to try and move past it.

In those cases you can still be supportive but don’t make the mistake of doing someone else’s job for them at the expense of your own. In general I think solution architects need to be ready to support delivery, but with as light a touch as possible. This assumes you’re on a fixed term project as opposed to a product team in which case it might be different.

If we look through a RASCI lens, typically architects are not responsible for delivery (outside of the architecture), they are only consulted or supporting.

How to be supportive in managing delivery? At first glance continuous delivery is a classic “people, processes and tools” scenario, so considering it through those lenses might be a good place to start.

Here’s some thoughts: things you might observe, and responses.

Observations that might prompt these questions:

Are there problems, is the delivery on fire? Investigate – what’s the root cause?
Are there issues working with teams / stakeholders outside of your control?
Are things sustainable? Are people working at a sustainable pace?
Is there too much work, are the deadlines unrealistic?
Do people have the right skills, and skill levels?
Do you have the right processes? Are the processes being followed correctly?
Do you have the right tools? Are they being used correctly?
Is there a bottleneck or some kind of resource contention?
How do you know the above is true or not – do you have the right visibility?

Potential responses:

Is it something the team can self-resolve through discussion – e.g. at a retrospective and by adopting some goals? (Don’t forget SMART).
Faced with a risk or issue? Can you: avoid it, reduce or control it, transfer it somewhere else, or do you have to accept it? The latter might mean making changes at your end even if the issue originates from outside.
Do you understand all the stakeholders involved – their motivations and priorities? Sometimes smooth management just means keeping the right people in the loop. Consider stakeholder mapping.
Bring in expertise from outside, probably on a temporary basis. Could be a formal engagement or informal. Sometimes someone from the outside will immediately see the issue which the team couldn’t see because they were too close to it. Sometimes advice from an outside (neutral) perspective helps get doubters on-board.
Do you need to escalate? Perhaps you need more staff, better technology or something else which you need to ask for.

Enquiry Questions

Posted on July 22, 2022 by adriankearns

TL;DR – Asking questions when you already think you know the answer still has benefits including relationship building, gauging expectations and navigating tricky situations.

I was in my first meeting with the client for a new consulting / architecture assignment. The project had been running for a while with one of my colleagues already involved. At one point I put a question to one of the customer representatives – but my colleague cut-in an answered on their behalf. It wasn’t totally surprising, since we had discussed the same subject in preparation for the call, so to a certain extent I knew the answer. Straight after the customer meeting my colleague and I were doing a debrief:

Colleague: “Why did you ask about [that topic]? I already told you [something relevant].”

Why? There’s several scenarios: relationship-building with the customer, getting on to the “same page”, verifying / re-verification information, or using it to then transition into more delicate topics. I’m going to call these “enquiry questions”. Let’s look at some different scenarios.

Relationship Building & Inter-Personal Connection

When you’re coming fresh into a already running project you need to get up to speed as quickly as possible – and not just the technical side, more importantly it’s about quickly establishing effective inter-personal connections with the people you’re working with. As an architect you are in a leadership position, and your success will largely hinge on your key relationships and how well you can leverage them.

In the consulting context, asking about some aspect of the project is a bit like showing personal interest – it demonstrates that your engaged and interested, which is exactly what key people want to see from you: that you are speaking their language, that you have a sense of the right priorities, that you’re not afraid to ask questions, and so on.

Asking enquiring questions also gives you an opportunity to gauge their response – how they react to a question can indicate how they feel about a given topic, where their priorities lie, and so on.

Getting on the “Same Page”

Being on the “same page” is metaphor for having a similar understanding of a given context, and being broadly like-minded regarding how that context will be treated. In a project or consulting context this must at least cover scope, objectives and priority – it’s critical for everyone that you and the customer are on the same page, otherwise it will lead to friction that will distract from getting things done.

Asking something we already know (or strongly suspect) allows us to confirm we are on the same page or expose gaps where we are not, allowing us to then start closing those gaps.

Reconfirmation

Personally reconfirming something directly means that you won’t get caught-out by information that is secondhand or out of date. It feeds into the “same page” concept and can also help transition towards more detailed and focused conversations.

Being an architect, or in any position of responsibility, means you need to be very attuned to the information you are basing decisions on – because if you don’t do due diligence on critical information and assumptions, it’s your fault if things go wrong.

Direct personal reconfirmation isn’t just risk mitigation, it also means you can tune in to any nuance – such as the customers attitude to the topic:

Are they clear and precise when they discuss it – implying they have a clear understanding that also gels with reality.
Do they show genuine passion or are they indifferent, i.e. how much you need to care about it might be driven by how much they care about it. Alternatively, low customer enthusiasm for something critical may indicate an issue which needs addressing.

Through no fault of their own, the people before you may have “listened but not heard”, so reconfirming things also about giving yourself a chance to dig deeper and be more precise.

Delicate Topics

Asking about something you already have some knowledge of can be a safe way to start a specific discussion – on the assumption that you already have some expectations around where the conversation will likely go – meaning that you are better prepared for it. Starting off with a conversation that goes well also helps to build immediate confidence and provide a platform for trickier questions.

Sometimes people need to feel that they are heard. Asking an open question can be a good opening for that, perhaps allowing a customer to air frustrations that need to be worked through. Asking the question in the right way can also indicate your position, which may in-turn help channel the type of response they provide.

Risks

Is there a risk you’ll look stupid asking something you should already know? Yes, but it’s usually small risk. Generally I’ll be asking the question for a reason, and part of the calculus will be a gauge of how the person is likely to respond – will they be annoyed or open.

A big factor in risk management, which you control, is how you frame the question. Done correctly you can use such questions as a lead-in to harder topics – assuming they are related and the segue is therefore natural. A simple way out is to literally say that you want to start by ensuring everyone is on the “same page”. Being new to the project usually affords you a certain amount of leeway, so covering old ground is not likely to cause any issues.

Roles & Responsibilities of the Solution Architect

Posted on August 3, 2021 by adriankearns

This post contains materials presented at the Geekle Worldwide Software Architecture Summit, Volume II (August 2021).

TL,DR: There’s not a lot of definitions out there regarding the solution architects roles and responsibilities – especially at a practical task / artefact / scenario level. So, based on my practical experience I’ve put together the following diagram / spreadsheet combo:

Solution Architect RACI v1.5 (png file)Download

Solution Architect RACI v1.5 August-2021 (xlsx file)Download

And if you need it, here’s the deck I presented which will provide some background:

Solution Architect Roles and Responsibilities – Adrian Kearns – Geekle WSAS Conference Aug-2021 (pptx file)Download

For those of you feeling like a more in-depth explanation, read on…

Problem Space

Knowing exactly what you’re supposed to do as a solution architect is mostly down to figuring it out on the job, learning from others, and what you negotiate it to be. It’s also hampered by inconsistencies that stem from four causes:

Variation in organizational size & maturity.
Variations in methodologies used.
Variable understanding of the solution architect role definition.
Organizational politics & human nature.

#1 deserves special mention, so let’s get 2, 3 & 4 out of the way:

Methodologies are often associated with (or dictate) various roles and responsibilities, so the methodology in use will often impact how architects do what they do, and even what they do.
The solution architect role definition tends to vary across cultural-geographical spheres and organizations (e.g. government vs private; where software / IT systems are the core business – or not, and so on).
Politics, e.g. where managers set-up fiefdoms that can sometimes impact who does what, within what scope, how they do it, and so on.

Whilst those three drivers are not uncommon, I’d argue the most consistent drivers are variations in organizational size and maturity. Size drives role focus – how specialized your role might be, whilst maturity drives role clarity – how well / consistently its understood.

Larger organizations tend to bring greater role specialization:
= Less need for you to “wear many hats”.
= More focused responsibility.

The more mature an organisation is:
= The more well developed it’s processes are likely to be.
= The more clearly defined your role is likely to be.
= Greater clarity on roles and responsibilities.

All of these factors will influence exactly what your role and responsibilities are, in a given engagement.

Current Art

Apart from specialist entities such as IASA Global, there’s not a lot of existing stuff out there that describes what a solution architect does, in terms of specific roles and responsibilities – especially if you want to get down to the task / artefact / scenario level.

First off, TOGAF, the popular enterprise architecture framework is… well… enterprise focused, not solution focused:

The Solution Architect has the responsibility for architectural design and documentation at a system or subsystem level, such as management or security. A Solution Architect may shield the Enterprise/Segment Architect from the unnecessary details of the systems, products, and/or technologies.
https://pubs.opengroup.org/architecture/togaf91-doc/arch/chap52.html

And that’s about as specific as it gets – apart from the TOGAF Architecture Skills Framework, which is generally good but it’s not very low-level and more focused on roles central to enterprise architecture.

SAFe goes further and looks like it has some good practical level of detail in terms of roles and responsibilities, based on its concept of “Solution Architect/Engineering“. I haven’t gone through SAFe training so can’t say what all the details are, but as you can see from the linked page, there’s a good level of practical detail.

The only caveat is that the guidance is SAFe specific; applying it outside the context of SAFe is definitely possible but will require your interpretation and adaptation.

My Model

The basis for my RACI model starts with the SDLC (Software / System Development Life Cycle), on the basis that:

Solution architects typically operate within the context of a project…
which usually delivers one or more systems / major system changes…
those changes typically follow an SDLC…
Therefore: SDLC drives a lot of what a solution architect does.

The gold elements in the diagram represent major concepts under which we can group specific tasks, artefacts and scenarios, for which we can define specific roles and responsibilities. The thread of gold elements in the lower half are directly related to the broad notion of SDLC (see slide 6 in the presentation deck):

Initial planning & analysis.
Development of the system / solution architecture.
Detailed design including API specifications.
Build, and the closely related areas of delivery methods and engineering.
All necessary testing including system integration, performance and security.
Release & deployment including data migration and roll-out planning.
BAU operations – the system / solution in it’s operational form.
System change – which may trigger new iterations of the SDLC.
Decommissioning when the system is replaced, which may happen as part of the implementation of its replacement.

Supporting those, at the top of the diagram, are a few major concepts outside the SDLC:

Current & Planned Constraints such as existing technologies, emerging systems and technology strategies.
The Architecture Practice you may be working within.
The Project / Product Construct you are primarily engaged with.

PaRCINo Matrix

Supporting the diagram is a matrix that describes every element on it, and the typical level of involvement you’re likely to have with it as the solution architect.

You’ll notice the matrix uses a type of RACI to define the solution architects level of involvement, but that it’s “PaRCINo” rather than “RACI”.

Pa – Participate – where you participate in completing the work.
R – Responsible – where you take the lead in completing the work.
C – Consulted – where your opinion is sought.
I – Informed – where you are kept informed of progress or anything of significance.
No – No involvement.

It’s worth taking a closer look at consult vs participate, especially in the context of solution architect, which is a senior leadership position within the construct of a project:

Consult implies you might have a degree of authority in relation to the subject, potentially including the ability to approve or endorse, or to at least lend significant sway to the subject.

Participate is similar to consult in that you might have a degree of authority, however, the practical application of that authority is (or should ideally be) through collaboration and driving consensus.

The challenge you face is that whilst you have responsibilities to your project team and business client, you also have responsibilities to your parent architecture team / chief architect / organizations enterprise architecture, and those two broad sets of stakeholders won’t always align. Where there is misalignment you will have to decide how to navigate them, and this may mean you have to assert your position more strongly than as a member of an egalitarian team.

Example: Domain Architecture & Breaking New Ground

The deck covers five examples from the model, which help to illustrate the roles & responsibilities of the solution architect, and how that can vary depending on circumstances. Let’s take one of those now.

This example actually covers two related concepts:

Domain Architecture – organisation-wide architecture for a specific domain, such as data, security, infrastructure and so on.
Breaking New Ground – scenarios where the development of a solution architecture requires the organisation to make major changes or additions. These initiatives may be outside the delivery scope of the project but still a dependency for it.

The connection between these is that projects delivering non-trivial systems, or major system changes, will often discover gaps in the organizations wider architecture / technical landscape. These gaps can be significant enough to block the project, and frequently impact one domain more so than others.

The precise role and responsibility of the solution architect in such cases depends on factors such as those previously mentioned (organizational size and maturity, and so on). What is clear is that such a blocker needs to be mitigated (such as by filling the gap or avoiding it), and that you are in a lead position of responsibility to navigate that.

What I find in practice, is that domain architects will usually want to work closely with you, because they will see your project as a means to deliver part of their domain architecture (because your project has budget and people / delivery resources). This gives you a degree of leverage. Some will also want your opinion.

Where the gap in question is a big one, and the organisation already has well developed plans to address it, your involvement might be somewhere in the inform – consult range.
If your project helped “uncover” the gap, or simply because you’re the first project to be in a position to actually do something about it, your involvement will at least be consult and is more likely to be participate. A good domain architect will look to you as someone to help share the load, and develop thinking around the practical delivery and operational details.
Where no formal domain architecture (or architects) exist, you might find yourself responsible, given that the gap still exists to imperil the project.

Geekle Meet-up Feb-2021: APIs and Microservices, Architecture Cheat Sheets

Posted on February 26, 2021 by adriankearns

I recently had the pleasure of presenting a couple of topics at a Geekle architecture guild meet-up: Modern API & Microservice Platform Reference Architecture and Architecture Cheat-Sheet. The decks and supporting materials are all below:

Modern API & Microservice Platform Reference Architecture

Presentation (pptx).
Presentation (pdf).
Modern API & Microservice Platform Reference Architecture (pdf)

Architecture Cheat Sheet

My thanks to my employer, Middleware New Zealand, for supporting me in this meet-up.

Please note, all the materials are licensed by either Middleware NZ or myself under a Creative Commons Attribution-Non-Commercial 4.0 International License. https://creativecommons.org/licenses/by-nc/4.0/

You are welcome to use the cheat sheet materials internally within your commercial organisation as long as it’s only used for internal staff development, and not offered externally for payment, and the Creative Commons attribution is honored.

https://www.linkedin.com/in/adriankearns

https://www.middleware.co.nz/

Four traps experienced solution architects fall into, and how to mitigate them

Posted on January 27, 2021 by adriankearns

Here’s my “four traps” deck (MS PowerPoint) from the Geekle solution architecture summit, January 2021.

four-traps-experienced-solution-architects-fall-into-and-how-to-mitigate-them-adrian-kearns-geekle-sa-jan-2021 Download

Why?

Experienced architects are not immune to falling into traps of their own making.

By looking at some specific ones we can be more aware of what we do, and why,
so that we can self-improve.

What’s In This For You?

Four traps I have seen architects fall into: their causes & consequences, plus some potential mitigations.

How to use this deck:

Evaluate yourself honestly against each of the causes. Seek feedback.
Consider the mitigations; action the ones you think will work for you.
Repeat – seek continuous incremental improvement.

Your first project – where to start?

Posted on January 27, 2021 by adriankearns

Here you’ll find my deck and the materials mentioned in my “Your first project – where to start?” presentation at the Geekle solution architecture summit, 2021.

This presentation looks at what to do on your first engagement as a solution architect.

Here’s the deck:

your-first-project-where-to-start-adrian-kearns-geekle-sa-jan-2021 Download

And here’s the underlying content: diagram and notes (see the PDF):

Solution Architecture Engagement Flowchart

solution-architect-engagement-flowchart-jan-2021 Download

This work is licensed by Adrian Kearns under a Creative Commons Attribution-NonCommercial 4.0 International License.
https://creativecommons.org/licenses/by-nc/4.0/
https://www.linkedin.com/in/adriankearns/

Quality Attribute Concerns for using Microservices at the Edge

Posted on August 13, 2020 by adriankearns

It’s often interesting to look at extreme system scenarios and see what can be learnt from them. This webinar (https://www.youtube.com/watch?v=Fz1BJTephWw) from the Software Engineering Institute (SEI) at Carnegie Mellon University, presented by Marc Novakouski and Grace Lewis, looked at some extreme scenarios and had some interesting takeaways.

Rather than regurgitate the entire presentation here, I’ll touch on the essentials and then discuss some points of interest. If you need further info have a look at the webinar – it’s pretty concise and the information is well presented, and well worth a look in any case.

What is “The Edge”

The edge refers to extreme system scenarios, such as those potentially experienced by first responders, people providing front-line humanitarian aid or combat personal – specifically where this is being done out in the field. For example: disaster relief and on-site coordination.

These scenarios are very far from the office or metropolitan ones most of us typically architect for: infrastructure (power and networking) may be unavailable or limited, limited computing power (maybe only what you can carry with you, on foot), conditions on the ground can be unsafe (natural disaster, enemy activity), and mission duration & scope may be unknown or very fluid.

Relevant System Quality Attributes

The following seven attributes were covered, based on experience gained through SEI’s Tactical and AI-enabled Systems (TAS) initiative:

Reliability
Survivability
Autonomy
Adaptability
Flexibility
Distributability
Openness

Many of these may appear as common sense once you think about the scenarios a bit, for example being reliable.

Autonomy is interesting. It’s the idea that systems need some degree of intelligence / autonomy so that it can perform some actions on it’s own whilst the user is dealing with mission or environmental concerns.

Modularity & Monitoring

What I took from the quality attribute discussion is that modularity & monitoring are key to several of them.

High reliability and survivability can be achieved when the architecture is modular – allowing components to be replaced e.g. the instantiation of a new microservice instance; but more than that – in this case a modular system can also imply an array of instances forming redundancy. This second point was one of the interesting ideas discussed – a systems ability to scale down (or in) and still operate, if the edge situation demands it.

Monitoring is obviously the mechanism that enables recovery – whether through user intervention or system automation – i.e. autonomy. Whilst autonomy can cover functional aspects it’s also useful for self-repair, working towards better reliability and survivability.

Complexity vs Reliability

The autonomy discussion threw up AI as a possible way of achieving autonomy, which to me implies a level of complexity that might conflict with the desired level of reliability, since the more complex a system is the more likely it is to fail, because the number of failure points and scenarios increase.

I was able to pose a question on this and the response was elegant and – you guessed it – came back to modularity and monitoring. What I took from it was this: Modularity is the key – the idea is basically to encapsulate areas of complexity so that they are self-contained and isolated as much as possible (think IoC / dependency injection and programming against contracts). Having safely isolated the complexity, you can then use simple and robust architectures to harness the power of these complex modules – e.g. Pub/Sub.

Finally, Microservices?

The majority of the discussion could apply to “edge” solution design at any level – the ideas discussed are not limited to microservices in their application. That said, microservices were covered albeit briefly (there’s only so much you can cover in an hour). Certainly microservices have many attributes and advantages that make them well suited to edge scenarios where reliability and so on are important.

Stealthy Bullitt Pump Compartment

Posted on April 28, 2020 by adriankearns

There’s a fair amount of usable space under a Bullitt’s cargo deck, space which I don’t think is often utilized to it’s full potential. A good candidate for making use of this space is small / thin objects which you want to have with you all the time – like your pump and puncture repair kit.

In this article I’ll summarize how I’ve designed and implemented a stealthy compartment for holding such things.

This solution aligns with my guiding Bullitt principles:

The Bullitt should be carrying stuff, not me: E.g. I should never need to wear a backpack.
Peak performance: the Bullitt is a lean machine, so unless restricted by a specific cargo load, the overall performance of the Bullitt should be maximized. That means nothing that limits speed, handling or cargo carrying.
Elegance: the Bullitt’s fit-out (locks, pumps, bike bags, etc) should be elegant. Elegance should be the product of following the first two principles, and likewise, by aiming for elegance I should achieve the first two principles well.

General Concept

The concept is to have a hinged compartment that runs along under the cargo deck. The obvious location is along the side opposite the steering arm – for easier access. Having it hinged at one end provides a good balance between accessibility and strong mount.

By utilizing the space under the cargo deck:

The exterior of the bike is not cluttered with stuff, which frees up space for other things (both on the frame in general, but also in terms of cargo carrying.
It is not directly visible in most scenarios, so not so obvious to casual thief’s.
It is protected from the worst weather.

Compartment Design

In my case, I only have a thin space available since I already used the central space under the cargo deck for my speaker system (see: Bullitt Boom Box).

I decided my compartment only needs to hold the pump and puncture repair-kit. I didn’t build it to hold my tools as well – for a number of reasons, one of which is that this is bit of an experiment: start off small and see how it goes.

Above is the compartment, detached from the Bullitt. On the left is the forward attachment point, through which runs a single bolt with a wingnut; this is how you release the compartment for loading/unloading. On the right is the top of the rear mounting hinge. This is permanently screwed to the cargo deck.

The long wooden board provides the main structural part of the compartment, joining the two attachment points, and provides something to secure the pump against. The rear box-like area holds the puncture repair-kit.

You’ll notice the gap at the rear, between the hinge and the side board, this is to accommodate the Bullitt’s front wheel brake and light cables that run along under the frame.

Mounting

The rear mounting point is immediately in front of the cross-bar, on to which the kick-stand presses when the Bullitt is parked.

Be aware that the sweep of the kick-stands pads will likely scrape the back of the pump compartment if it’s too close. Also make sure there’s enough room for any cables to run past.

In terms of the cables, at the front of the Bullitt these should be located to the side, before the run up to the front forks. This means that there should be enough room for the front mounting bolt on the inside of them. I then have mine running down the side of the speaker enclosure, in such a way that they are secure but with enough flex to run comfortably along the outside of the compartments long side board. They then exit through the rear of the compartment. Make sure that the compartment doesn’t foul on them with opening or closing.

The front attaching position is well forward – just behind the front cross-bar.

Images: left – the rear mounting bolts. Centre – with the compartment removed, showing the bolts sitting in position ready to take the compartment. Note the location of the kick-stand (in down position) and the cables running around the back of my custom speaker enclosure. Right – photo taken from a similar angle, showing the compartment attached.

There are three mounting bolts, all of which are flat-head and counter-sunk. When mounted, this type of bolt head sits flush with the cargo decks surface, so there’s zero impact to cargo.

My cargo deck is made from 7mm plywood. This isn’t especially thick, but it will safely hold the counter-sunk bolt head as long as the bare minimum of counter-sink is used (so that as much wood as possible is retained, to hold the bolt head). Part of the reason I think this is safe is because the load the compartment carries is very light.

Note: depending on your design, I suggest you periodically check the holes, and the wood around the bottom of them, to ensure there’s no wear or weakening – you don’t want the compartment to come loose whilst riding.

The forward bolt:

The hole in the deck only needs to hold the bolt in position, the bolt is held fast by the wingnut used in combination with a flat washer and a spring washer.
The diameter of the hole in the compartment is such that it slides easily on/off the bolt.

The rear mounting bolts – I’ve used two slightly different approaches for these:

The holes in the deck only needs to hold the bolts in position, the bolts are held fast by the compression due to the compartment mounting. The same diameter is used for both bolts.
The compartment hole for bolt-A is tight, so that the bolt’s thread bites securely into the wood. Be very careful not to over-tighten otherwise you risk striping out the wood that holds the bolt secure.
The compartment hold for bolt-B is not tight, the bolt slides through and is fastened by a hex type locknut.

Why the two different approaches? Basically it’s bit of a hack and an experiment. making the wooden hinge was much trickier than I anticipated – the one you see is about the third attempt. The wood used is strong enough but I managed to break the first one whilst working on it – so I was mindful not to over-stress the wood.

I also wanted to maximize the strength of the hinge, which meant having enough thickness to secure the metal hinge, side panels and so on. Using this combination seemed to work the best based on my first two attempts.

Bolt-A provides the most timber for strength of the unit, whilst bolt-B provides the most secure fastening due to the locknut.

In Closing

The position of the front bolt is such that it’s fairly easy to access the compartment, but it is a little bit fiddly to re-attach. It’s definitely convenient enough for occasional use but it’d be nice to have a slicker design if you wanted really frequent access – and don’t forget to manage the Bullitt’s cables.

The pump is secure sitting in the compartment, but I also use a piece of rubber to tie it to the side board so that there’s no rattle whilst riding.

I’ve had this in operation now for around 6 months and all’s going well.

If you don’t have a speaker system under your deck: (a) you should get one :P, and (b) you will have more flexibility in terms of your compartment design and what you can accommodate in it.

Bullitt Boom Box

Posted on April 1, 2020 by adriankearns

Bullitt’s are awesome, but sometimes awesome can be more awesome when you’re playing your favorite tunes… and what better way to do that than by building a stealthy speaker system for your Bullitt, of course!

Speakers – not seen, but defintaley heard.

As you can see – the speakers sit no lower than the steering arm, and do not have a significant impact on the cargo deck or load carrying.

The assembly to the right og the picture is a seperate hinged bracket that holds my pump, and puncture repair kit – always handy, but totally out of the way.

Images: Various views of the speaker enclosure. (Right) a stealthy pump and puncture repair-kit holder, next to the speaker enclosure.

The premise of the design is that because we tend to ride bikes on hard surfaces (i.e. sealed road) the sound waves will bounce off the road and be audible. I can confirm through experience that this is definitely the case – more on that in a later section. As such, the speakers are mounted under the cargo deck, facing directly at the road.

This location is significant: it means the cargo deck is virtually* unconstrained by any sound system paraphernalia – so it won’t interfere with any load; the largest part of the system – the speakers and their enclosure – is neatly tucked in under the cargo deck, meaning there’s no impact on the bikes performance; and it’s visually obscure, so not an obvious target for theft. So you can essentially leave it on the bike all the time: simply jump on, turn up the vibes, and turn heads.

* The only impact is the bolt-heads that protrude above the deck, and even then there are mitigations. For me these are very small so there’s no practical impact on load carrying. See the “Speaker Enclosure > Bolt-Heads and Their Impact on Load Carrying” section below for more info.

Disclaimer: the ideas and information provided here are provided “as-is”, no warranty is provided or implied. Building a system such as the one described here involves various risks, both during implementation and operation. If you damage yourself, someone else or any property, through directly or indirectly being influenced by this content, that is entirely your responsibility.

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. Want to commercialise this? I’m open to discussion – get in contact.

Above, the main on-Bullitt components, including speakers, amplifier and battery.

The Recipe

You will need:

Amplifier.
Speakers (water resistant).
Speaker cabinet / enclosure, with mounting nuts & bolts (assumes you have appropriate mounting holes in your Bullitt’s cargo deck).
Battery.
Battery Charger.
Battery charge/voltage meter.
Speaker cable (approx. 1.5 metres x 2).
Speaker cable bungy (x2).
Old bicycle inner tube (speaker cable protector).
Various cables including: battery to amp, battery to charger.
Carry pouch for amp & battery.
Connectivity between the amp and your phone / music player.
Speaker rear cover-plate (optional).

Amplifier

The heart of the system is the amplifier – so choosing this first is recommended. You want an amp that has good power output, and is relatively small and light-weight. Another critical consideration in amp selection is how you’ll power it (i.e. what it’s power requirements are – more on that later).

You may want to consider amp and speaker in combination, since it’s important to ensure these are well matched in term of power per channel and speaker impedance. In my case I bought the amp first, based on guidance from a friend and colleague of mine, Steve*, and selected other components based off that.

* He used to work for Sony fixing electronics. One day a VIP client brought their favorite Walkman in for repair – my colleague was selected to repair it. The client was Princess Diana. So, needless to say, I trust his judgement. 🙂

Steve recommend a digital “D-Class” amplifier, because the overall characteristics of these amps are well suited for use on a bike: they are lighter and more robust that a valve-based amp, and have very efficient power usage (which is important as we’ll be running off a battery).

The actual amp I use is: “TPA3116 Mini Power Amplifier ISSC Bluetooth HIFI Stereo Audio Digital AMP 50Wx2”, purchased from China via e-bay.

Amp with my v2 switch strapped to the side.

Images: (Left) Promotional photo. (Middle) Rear of the unit, with hand for scale. (Right) Mounted on spacer-board, and with power on/off switch.

Amp specifications:

Work efficiency: 90%
Rated output power: 2 x 50W
Frequency response: 20Hz to 20KHz
Operating voltage: DC18V to DC24V
Maximum output current: 3A
Bluetooth receiving range: 10 meters
Size: depth 108* width 90* height 39mm (without antenna)

References:

https://www.ebay.com/itm/273216712656

Speakers

I discovered that Pioneer make “Marine” speakers – i.e. for use on boats etc. These are designed specifically to operate in wet conditions; the manual says “Marine use, water-proof design … UV and corrosion-resistant design”.

I have a pair of these, specifically: TS-MR1640’s, which are a good match for the amp. I had intended to get the TS-MR1600’s but StreetSoundz (where I bought them) kindly offered me the next model up for the same price 🙂

As mounted, in v2 of my enclosure.

Exterior view (unmounted).

Typical grime build-up after a few months of summer usage.

Images: (Left) Spec’s. (Inner-left) Rear of a speaker, as mounted in the enclosure. (Inner-right) The business end of the speakers as mounted (note, enclosure unmounted in this photo). (Right) Speaker with cover removed, showing grime build-up after several months use.

Another great thing about these speakers is that they are a decent size – about 6 inches, so not too small yet fit nicely under the Bullitts deck, which is key to the overall design.

You could use non-marine speakers, but it would be somewhat limiting; if you’re like me, and operate your Bullitt in all weather conditions, then the marine speakers mean you can still pump out the vibes even in the rain. I’ve painted mine using some oil paints I had lying around (didn’t have black), mainly to make them less visible.

TS-MR1640 Speaker Specifications:

Maximum music power: 100W.
Nominal power: 25W.
Impedance: 4 Ohm.
Woofer diameter: 160mm (~6 Inches).
Sensitivity: 90dB.
Frequency response: 30Hz – 20kHz.
Weight: 790g (per speaker).
Depth: 56mm.
Speaker cover: 28mm high, 168mm diameter.

References:

https://www.pioneer-car.eu/eur/products/ts-mr1600

Speaker Enclosure

I made the speaker enclosure myself, mostly using pine and plywood. The enclosure you see here is my second version.

Notice the four holes in the middle: the outer two are an attempt at bass-reflex; the inner two are bolt holes – but you’ll only need one.

The curved pieces of plywood are to strengthed the speaker mounting screws.

Images: Various views of the speaker enclosure (unmounted). Note the additional ply (also 7mm) around the speakers to provide a secure anchor for the speaker mounting screws.

The essentials of the design are:

A plywood (7mm) plate to mount the speakers. Note I have used a second layer of ply inside the enclosure to provide extra strength for the speaker mounting screws – the last thing you want is for them to come loose through all the shocks and vibrations associated with riding.
Light-weight and strong sides/ends. I’ve used untreated 60mm x 10mm clear dressed pine.
Bolts running vertically through the enclosure, so that the open / rear side of the enclosure is held flush against the bottom of the main cargo deck – which is also 7mm ply.

My design is like this:

You’ll notice that the middle bolt hole is off-center. The reason for this is the lack of space between the speakers along the center-line, plus the presence of the Bullitt’s first central cross-bar. Initially I had a symmetrical design – 4 bolts with two in the middle, but I have since dispensed with one – three bolts in total is enough.

Speaker Enclosure Design - Measurements 2

Speaker Enclosure Design - Measurements 1

I have a custom 7mm thick plywood cargo deck on my Bullitt, plus the enclosure’s 7mm speaker plate, plus the 60mm high sides, which is an overall depth of 74mm. The enclosure is bolted to the deck using 80mm M6 bolts and Stainless Steel Nyloc Nuts. The initial assembly is fairly tight on the bolts, but you’ll find that the plywood is somewhat malleable, so you will get secure assembly. (You’ll want to keep the bolts short as possible so they don’t protrude down too far and catch on anything). After 8+ months use I find that the bolts will happily extend past the top of the nut, as the plywood molds to the pressure of the washers/nuts. Initially you might find they end of the bolts only go to flush with the top of the nut – which is fine if you use lock-nuts. Just check them regularly to make sure they are not coming loose.

Plywood is an excellent material for the plates, as it’s light-weight and strong; clear pine is fine for the sides and ends – again because it’s light. 10mm provides enough strength to maintain integrity, and because the design has the bolts passing through the plywood – and taking all the weight – the pine is not really load-bearing.

The Bullitt has two central horizontal cross-bars running across the cargo deck – the length of the speaker enclosure means you’ll need to cut parts of the sides away, so that the enclosure fits snug around this cross-bar, and flush against the bottom of the cargo deck.

You’ll notice the angular joinery at one end; this isn’t part of the acoustics, it’s pure decoration, I thought a jet engine air intake type of look might be cool. All the joints are sealed internally with a wooden sealant from the hardware store.

Speakers – not seen, but defintaley heard.

Note the cut in the enclosure to allow for the cross bar. You can also see the nyloc nut of the front mounting bolt.

View from the rear, showing the speaker cable bungeed to the frame.

Images: Various views of the speaker enclosure. (Middle) Note the recess that accommodates the frame’s cross-bar, allowing the enclosure to be flush against the bottom of the cargo deck. I’ve wrapped recycled inner-tube around the steering rod for protection from scratches. (Right) note the inner-tube protecting the speaker cables.

Bolt-Heads and Their Impact on Load Carrying

The bolt heads are only a few mm high, which (in my experience) aren’t an issue for any loads. I often use large fish bins; the design of these includes a small rim around the bottom, meaning there’s a thin recess underneath them – which is enough to accommodate the bolt heads.

You might find them an issue though in some specific cases. There’s a few mitigations you can use, some of which affect the design:

Use packing blankets or some other soft material to protect your load.
Make a solid spacer that acts as the cargo deck, which has holes cut out for the bolt heads. This could be a separate piece of wood which you use as needed or you could build it into/onto the deck as a permanent fixture.
Use bolts / screws that result in a flush finish with the top of the deck. Note: you’ll want to make sure there is enough strength to hold the bolts even after months of use / wear & tear. I don’t think 7mm ply is safe to do that.
Use a different mounting system – e.g. screwing the enclosure directly to the bottom of the cargo deck. Note: this will make maintenance a lot harder, but not necessarily protect you from thieves.

Acoustics

I’m not an audiophile, so you may want to do your own further research the overall acoustics and the designing of speaker enclosures.

There’s no doubt that some types of music sound better than others. I tend to prefer Techno, Ska and Roots/Reggae/Dub; some Classical can work well too (a spot of Rostropovich, perhaps), and some Pop. Interestingly I’ve found House music doesn’t fair quite as well. Your mileage may vary.

The sound is generally clear, but some wavelengths can get a bit lost.

Because the speakers face into the ground, sound is emitted in 360 degrees. This means that you provide some fun and vibes for those around you – whether they like it or not. Interestingly, it also lets people know your coming, which can be useful form a safety perspective – more on that in the “Operation” section.

References:

Battery System

This is a huge and complex subject, so I’ve posted the details separately here: DYI 18650 Battery Pack

Speaker Cable

You’ll need two lengths, around 1.5m each. This runs from the amp, mounted in a pouch hanging off the back of the cargo deck frame, down the frame and along under the main frame to the speakers.

Do yourself a favor and get good quality cable.

Speaker Cable Bungy (x2)

Use these, or something similar to lash the speaker cable to the frame so that it doesn’t foul on anything. I have one on the side vertical, and one under the deck.

Old Bicycle Inner Tube

I use an old bicycle inner-tube to protect the speaker cables from getting scratched / cut, sunlight, and it’s a lot less obvious than speaker cable (i.e. theft) – mine was bright copper in a clear rubber seal, so bit of an attraction.

Various Cables

As with everything in do-it-yourself systems like this, you’ll need to ensure you can connect everything together. These are also covered in the DYI 18650 Battery Pack post.

Carry Pouch for Amp & Battery

Army surplus stores are great for random bags and pouches, usually at good prices. The bags I have are old rifle ammo pouches – the dimensions are perfect for mounting on the read cargo frame of the Bullitt, and holding the amplifier and battery. The canvas is nice and thick (protective). They are rated as “shower proof”, the canvas lids have sides that provide decent protection and clips for holding them down.

How you mount the amp and battery is entirely up to you. What I like about the pouches is that they fit perfectly at the rear of the frame – which means the volume knob on the amp is easy in reach, so I can change volume even when riding.

The top cover of the ammo pouch, attached to the Bullitt Frame using bundy cable – to help minimize vibrations.

How the battery and amp usually sit inside the pouch. Note the light-weight wooden board that the amp is attached to – this helps position the amp, and encure cables are not crushed.

The amp, positioned for easy access to the volume control. I removed the volume knob as it was too easy to bump. Battery not included in this photo.

Battery pack connected to the amplifier.

Images: (Left) Army surplus ammo pouch. (Inner-left) The amp and battery, as it sits within the pouch. (Right) The amp in its operational position; volume knob removed to avoid accidental volume change. (Right) Battery and amp.

For my design, I have attached the amplifier to a light-weight wooden board, which serves two purposes:

As the ammo pouch is bigger than the amp, it pushes the amp up towards the top, so that the amp is easy to access, even when riding.
By pushing the amp up, it also helps create a space where cables can sit without getting crushed.

The amp is basically just taped on to the broad, but if you look closely at the photo’s you’ll notice a small piece of wood bolted to the main board, which holds the weight of the amp when it vertically in the pouch.

Lastly, think about how you’ll mount the pouch to the Bullitt – in terms of managing shocks and vibrations. I’ve used bungee cord to suspend the pouches in place, to help minimize vibrations and shocks.

Speaker Enclosure, Rear-Plate

This is optional. It’s purpose is to cover the back of the speaker enclosure, if you want to use the speakers off the bike. The plate serves two purposes: protects the speakers, and acts as a baffle so that rearward generated soundwaves don’t distort those emitted by the front.

Amp-Smartphone Connectivity

This really depends on the amp; I had intended to simply use an old-school style lead: 3.5mm headphone jack to twin RCA connectors – RCA connectors are very common on amps. The amp I got also had Bluetooth, and I’ve ended up just using that exclusively. If you need them, cables like 3.5mm jack to twin RCA should be easy to buy off the shelf.

Operation

People

The most interesting point I’ve noticed is how people around you react, because the idea of audible music coming from a bike is not one most people are familiar with.

In one case, I was cruising along the waterfront in a zone which is mixed pedestrian and cyclists, next to this is a road. I was coming up behind an older gentleman, and as he heard the music (some techno) he instinctively looked towards the street – no doubt expecting some “youths” to go cruising past.

I’ve also sat at the lights with pedestrians nearby looking around, unable to sense where the music is coming from.

There’s also an interesting safety angle. There’s been a few times, especially around the CBD where pedestrians have been just about to step out in front of me – and they’ve heard the music and stopped.

Battery Management

Managing the battery charge is pretty critical – naming not letting it run down too far. make sure you check the behavior of your amplifier – as I mentioned above, mine has a small current draw even when switch “off” – approximately 0.025A. If you leave them connected to long you can risk discharging the cells to unsafe levels. Having some sort of protection is a good idea. I use an additional switch to completely break the battery-amp circuit, and keep in the habit of using it. Whilst I haven’t done extensive research on this, it’s likely you can find or build basic BMS’s that would cut-off in low voltage. The basic voltmeter shown above has a siren built into it, which will go off once you configured voltage is reached – but it has to be plugged in all the time (itself a very small power drain), and it’s only useful if you can hear it (so if you’re not in earshot…).

More info in the DYI 18650 Battery Pack post.

Security

One aspect to carefully consider with the design of your system is security. You will need to judge for yourself what the risks are, how likely they are, and how to deal with them.

The speaker enclosure design is a key component of this as it’s easily the single largest component. How it balances convenience and robustness with security is a key point to consider. My line of thinking is that the space under the cargo deck is perfect because the speakers are effectively out of sight; and out of sight = out of mind. You may want to be careful in how you operate – e.g. having music blaring out, and then turning the sound system off in a really obvious way, may attract unwanted attention.

I like the straight-through bolt design as it’s very strong; having the speaker enclosure come off whilst you’re doing 40Km per hour downhill would be… sub-optimal.

The other bonus is that it’s relatively straight-forward to remove the speaker enclosure – i.e. for cleaning and maintenance, or to use in the traditional sense – pointing directly at you and your friends rather than bouncing off the road.

My assumption is that as long as most people don’t know exactly where the speakers are, it’s relatively safe. There’s also the issue of needing the right tools, and time, to remove them.

Same logic applies to the amplifier and battery. You can design a system that is less obvious, and relatively less secure once it has been detected, or a system that is more secure but perhaps less convenient. Its really up to your personal preference.

Big Shout-Outs to:

Pete for building many of the version 1 electronics – battery, cables, etc, and advice on the battery design and charging approach.
Steve for the amplifier selection and other advice.
Street Soundz for providing a good advice and price on the speakers, speaker cables and RCA connectors: https://www.streetsoundz.co.nz/