Wedding IT Development to Business Success with DSDM - SPA Cambridge talk 14th May

Blogged under Uncategorized by Mark Dalgarno on Monday 28 April 2008 at 9:43 am

The next SPA Cambridge talk will be held from 19:00 on 14th May at Anglia Ruskin University, Cambridge (UK).

Something New Something Old - Wedding IT Development to Business Success with DSDM - Steve Messenger, Napp Pharmaceuticals

Synopsis

Agile development is seen as something relatively new. In fact many of the practices expounded have been around for quite a few years. 10 or more years ago IT professionals were, of course, experiencing many of the issues that agile methods hope to prevent - the wrong system being developed in isolation from the business and taking costing much longer than planned. In an attempt to counteract this, and also prevent a movement into a chaotic, hacking world, 12 professional bodies, including companies such as IBM, got together to create an agile method which also ensures quality and control. That method is DSDM Atern.

This talk will discuss:

  • The Problems DSDM Atern helps to solve 
  • The key working principles within DSDM Atern and how they merge nicely to the Agile Principles 
  • The key techniques DSDM Atern offers to ensure timely delivery of solutions that truly meet business needs and provide business benefit. 
  • How DSDM Atern is a good fit with Project management tools such as PRINCE2 and development methods such as XP. 
  • How DSDM Atern can be used together with the PMBOK. 
  • Some case studies of projects the speaker has carried out, with both the successes and areas to watch out for. 
  • The applicability of the method to offshore projects - the subject of a recent DSDM Atern white paper co-authored by the speaker.
  • The future for DSDM Atern and Agile.

The meeting is free and open to all but preregistration is required - please visit http://www.bcs-spa.org/cgi-bin/view/SPA/DSDMAtern for more information and to sign-up

Performance of Code Generators and M2M

Blogged under Code Generation, Software by Mark Dalgarno on Monday 21 April 2008 at 8:41 am

Steven Kelly has just written an interesting post about the performance of various code generators.

The basis for the post is an IEEE article comparing the performance of Eclipse MDD tools and Ruby DSL tools on a model with 40 classes and 50 inheritance relationships with each class having around 6 attributes.

He then compared the results with MetaEdit+ (his company’s toolset) and found that MetaEdit+ is over 30 times faster than Eclipse, and over 20 times faster than Ruby. He atrributes this difference largely to MetaEdit+ not having an M2M phase.

Interesting reading for code generator designers…

When Good Architecture Went Bad at ACCU 2008

Blogged under Architecture, Software by Mark Dalgarno on Wednesday 16 April 2008 at 6:20 pm

It’s 2 weeks to the day since I stood up in front of just over 30 people to lead my session When Good Architecture Goes Bad at the ACCU conference.

My plan was to present some examples of architectural decay, to collect some examples from the participants and to explore how things could be improved. I was particularly interested in the value and cost of work done to prevent architectural decay. It seems that developers agree that preventing such decay is a good thing but I was hoping to collect some examples that could make the financial side of things a little clearer as it seems to me that there is a disconnect between developers and managers in this area. It wasn’t too clear from the session description in the programme that this would be a workshop so first up I offered anyone who just wanted to sit and listen the chance to leave the room - there were no takers…

After a few introductory slides the first exercise asked participants to discuss examples of architectural decay from their real-life experience. I collected these on a flipchart:

Examples of architectural decay

  • A single class used as a dumping ground
  • Cancerous wart – ever increasing coupling between modules, packages etc.
  • The number of programming languages used on the project increasing over time
  • New interfaces added over time, but old interfaces still maintained (and never deprecated)
  • Lots of code clones  (copies and near-identical copies)

I then presented some examples of architectural smells (problems in package/class/subsystem/layer relationships, overgeneralization etc.) and whiffs (subtle smells) e.g. no one on the team can tell you what the intended architecture is, time to implement changes increases etc.). The second exercise asked the group to come up with their own examples:

Architectural smells associated with these (and other) examples:

  • Mismatch between documentation and software, mismatch between comments and code
  • No clear responsibility for the architecture
  • Implementation = specification
  • Use of a proprietary language compiler
  • Pile of s**t from the start of the project
  • Clone and own as the main way of doing reuse
  • Insufficient decomposition
  • Knowledge of architecture held by a decreasing number of people

The bulk of the session was taken up by two case studies. In Case Study 1 an outsourced project had run into problems over a period of years. The company detected a considerable decrease in productivity over this time and the participants were asked to decide whether architectural decay could have caused the decrease in productivity and what they thought of the company’s proposed solution. (If you’re want to try this at home first you can download it as Software Architecture Decay Case Study 1 (PDF) from the Software Acumen web site.)

Participants’ observations included:

  • Communication needed to maintain architecture
  • Management needed to maintain architecture
  • Someone required to shepherd the new team before they can get going
  • Insufficient knowledge transfer process when software first outsourced.
  • Team selection is important when assigning new roles.
  • Unclear when productivity decline happened. Why didn’t company pick it up sooner?
  • Who owned the software & the architecture – tests.

A second case study looked at a situation where a software system had been developed in 3 separate sites but the company had just closed two of the sites. Participants were asked whether moving maintenance to a single team at one site would cause the software architecture to decay. (Again, if you’re want to try this at home first you can download it as Software Architecture Decay Case Study 2 (PDF) from the Software Acumen web site.)

This time participants’ observations included:

  • Fewer people, resourcing could contribute to decay
  • Knowledge transfer / must learn new bits - again could cause problems
  • Subtle differences between architectures of the different parts could cause problems - initially it’s a comprehension task
  • Domain expertise was lost when two sites closed - could indicate significant loss of architectural knowledge
  • What were the future plans - is there an implied refactoring?
  • This situation needs management to go into the project with open eyes - non one was convinced this was the case
  • There might be an urge to change the architecture which could be risky
  • Not-invented here / cultural differences could lead to problems
  • The motivation of the new team was questioned, skills drain could occur
  • What was the background motivation for the change? This could set the tone for future work.

I then asked participants to travel back in time and come up with some ideas for maintaining architectural integrity in their previously noted problem projects:

  • Encourage people not to do clone and own – but how to do this?
  • Change the development process (again how viable is this, would management back it?)
  • Embed a culture of refactoring (again but what if management won’t allow it?)(and note - some developers may want too much refactoring, be too keen to rewrite)
  • Visualize the technical debt – detect architectural decay using tools
  • Make responsibilities clearer.
  • Add automated (architectural) checks.
  • Cancel the project (earlier).
  • Rewrite (earlier)
  • Kill the architect (as noted by SPA 2008 participants also)
  • Spread architectural knowledge.
  • Spread the architectural work.
  • Have frequent communication between whole group – up to three times a day.

The session covered some of the same ground as Tom Gilb’s keynote ‘Agile Methods Lack result management’, Steve Love’s session on ‘Snowflakes as architecture’ and (it later emerged) Dirk Haun’s session ‘Rewriting not recommended’.

Slides for the session area available at When Good Architecture Goes Bad (PDF).

Snowflakes and Architecture - Layers considered harmful - Steve Love at ACCU 2008

Blogged under Architecture, Software by Mark Dalgarno on Friday 11 April 2008 at 10:02 am

Steve Love’s session at the ACCU Conference was billed as taking a “suspicious look at the traditional layered architecture, and suggest[ing] some ways it can be improved upon, resulting in an “architecture” that resembles a snowflake more than it does a cake.” Attending his talk fitted in with my plan to focus more on the architecture sessions on Day 1 of the conference.

Steve was straight in with the boot beginning by noting that the main difficulty with layer diagrams is that the diagrams are often only what you get. (More marketechture than architecture)

Layers are created to separate concerns but often business logic leaks into different layers (more on this when I blog about my ‘When Good Architecture Goes Bad’ session) – this leads to untestable code due to unintended dependencies between layers e.g.  the business application that must be tested with a GUI. Another problem is that sometimes a layer is there only to pass data through to lower layers.

Steve’s talk described a different (granular) approach to making a more adaptable architecture by design and looked at the mutual influence of architecture and design.

Robert Martin notes that bad design is rigid, fragile (knock-on effects of local changes) and immobile (can’t move things around for reuse, can’t disentangle software). Booch (1996)  suggests that instead of layering,  we arrange applications into smaller granule components that communicate together. So good design is cohesive, decoupled & layered. But Steve questioned whether these are enough…

What a user sees as quality is different from what developer sees as quality. (MD - In fact there is often a similar disconnect between different groups within a development organisation and this can also lead to problems.) His claim is that simplicity in software architecture is key, is easy to measure, but is not straightforward to achieve.

To illustrate what he meant Steve introduced concept of Dependency Horizon that is the number of steps you need to bring in for a particular dependency (think of it as the number of steps between components, packages or modules if you will). A far dependency horizon can become hard to manage and can increase chance of circular dependencies (not good).

Decoupling (e.g. through intermediate objects or interfaces) improves maintainability. This can be done by identifying abstractions and pulling out pure interfaces. This enables the dependency horizon to be shortened and so maintainability is improved.

A key concept here is the Dependency Inversion Principle

  • High level modules should not depend upon low level modules. Both should depend upon abstractions.
  • Abstractions should not depend upon details. Details should depend upon abstractions.
  • This allows abstractions to depend on each other, but not on their implementations.

DIP is layering in the small.

The Singleton pattern came in for a lot of bashing in Steve’s talk. Singleton is the counter-example of DIP – it is the antithesis of detail depending on abstraction. The pattern has lots of problems – hard-wired dependency, dependency from within, testing is difficult, rigid/fragile/immobile (see above), unpredictable ownership, unpredictable lifetime, handling when multiple threads present. Conclusion – Singleton pretty much violates all of Martin’s design principles.

Steve’s solution is Unsingleton (MD - or perhaps antiSingleton) – code should work with what it wants, don’t let it take it for themselves. What client code doesn’t know about the implementation of a service can’t hurt it.

Steve’s solution to many of the problems was to Design to an Interface. This can be realised in different ways in different technologies:

  • Interface keyword in some languages
  • COM/CORBA - IDL
  • C++ Pure Virtual Base Class
  • Duck-Typing (ducking the whole idea of typing) - C++ Templates, Ruby, Python
  • C# and Java Generics

The key thing about interfaces is their substitutability (Liskov substitution principle).

  • Inheritance is the tightest form of coupling possible. Inheritance – polymorphism by dynamic despatch (the conventional model) – virtual functions (AKA the inclusion model of polymorphism).
  • Genericity – polymorphism by generics – (Duck typing) – Containers, Iterators and algorithms (strategy), Traits and Policy classes is better.
  • Overload – polymorphism by overloaded functions – member function overloading, global functions and operators. Can be particularly powerful in creating interfaces. (MD – this resonated with my experience in Common Lisp.)
  • Cohersion – polymorphism by conversion, implicit casting, constness. (MD – again resonated with my Common Lisp experience.)

Testability is an essential property. (MD - A quick aside - When I first started programming I didn’t think about this very much - we weren’t taught it and nobody around me considered it as a property that could be designed in. As I read more about software development I began to take more notice of this and began to look for testability in design documents. Debuggability is a related concept that supports testability e.g. I built a logging add-on for some quite complex database code I inherited and this really let me get to the bottom of some quite complex client-server interaction problems.)

Testability principles:

  • You must be able to Test independent parts independently
  • Interfaces = Substitutability – if software under test depends on database etc. then change it so that it must depend on interfaces (to the database), this lets you mock (or replace) the DB
  • Substitutability underpins Mockability (Mock objects). Gives you ability to mock out services that are only available on target device (pda, phone) and do debugging on PC (for example).
  • Design to an interfaces

Parallel Development

  • Working to interfaces enables parallel development, continuous integration, testing (again)
  • Also supports outsourced development
  • Adaptability – when using interfaces plugging in a new component just like an existing one is trivial. (MD – I did have a question in my mind about how much effort and knowledge is required in order to develop these interfaces. Does one need to have written the interfaces a few times in order to get them right?)

Flexibility, Generality and Reuse

  • The false idols of OO?
  • Usually done with inheritance but these led to big class hierarchies which were unusable rather than reusable. (I touched on this in my session too).
  • Interfaces provide the means of reuse. - Component architecture provides the means of flexibility – make stuff talk to a wire feed rather than the UI (say)
  • And Generality - Can be reused in different context

Fat interfaces are less useful than small interfaces since they bring in unwanted dependencies. Must design them according to what client code wants, not what it can use. Small, specific interfaces allow better reuse.

Dependency Injection – promises the idea of true decoupling. Spring etc. sometimes a good idea – but sometimes a sledgehammer to crack a nut. Picked up by people wanting to decouple things but they often miss lighter-weight solutions. SOA does give decoupled design but needs thought in Steve’s experience.

Parameterize-from-above is the solution. (Don’t call us, we’ll call you). All about separation of concerns. But it still requires programmer skill to put it all together (MD - what software doesn’t?).

Alistair Cockburn propose Hexagonal architectures – these promote the idea of the application as a service. (With ports and adapters). Port defines contract for adapters – this leads to pluggability. (MD – are interface technologies well-enough defined to support this in the general form? How much effort is required to create these entities?)

Making adapters have their own ports leads to software as a collaborating set of components. Steve notes it’s an attractive design but that it requires discipline and can lead to circular dependencies. At the back of his mind he did seem to be worried about the danger of replacing spaghetti code with spaghetti interfaces. One suggestion was to try and organise interfaces into layers to help. The key here he suggested is to break layering down into its constituent interfaces. So that we still layer components but we don’t make them depend on being in a layer. This allows decoupling within layers.

So - what about the snowflakes? Well, these are pretty hard to describe without a picture, which I don’t yet have. Look out for Steve’s slides on the conference web site adn you’l get the picture (pun intended).

Agile methods lack result management – Gilb keynote at ACCU 2008

Blogged under Software by Mark Dalgarno on Wednesday 9 April 2008 at 8:55 pm

Tom Gilb kicked-off the ACCU conference on Wednesday with a controversial keynote, for some,  arguing that Agile methods do not have enough focus on delivering value to the people who pay the wages.

Tom began by claiming that agile methods are too self-centred in process and methods. This makes them good for focussing on programming tasks but they need to be supplemented with other methods in order to manage value delivery.

Perhaps unsurprisingly Tom suggested his own EVO method as the ideal method for doing this (although he noted that you could also use DSDM or RUP – both of which look more closely at value than Agile methods). Tom cited several recent examples of work done by Ryan Shriver (of Dominion Digital) on wrapping SCRUM with an EVO envelope. Ryan’s findings provide evidence that this combined approach provides both a method for managing value delivery and a method for managing programmers.

A fundamental problem with XP and SCRUM (for example) is that they don’t provide guidance on the highest-value / lowest-cost work packets. They have no alignment with higher-level business goals, argued Tom, and consequently no measurement can be performed with respect to business goals. (This provoked some audience heckling but of course a full debate couldn’t be carried on in the keynote).

Tom proposed the following framework for making smart decisions:

  • Measure progress towards goals – burn rate (stories) isn’t a good metric
  • Get a better understanding of time, budget, people
    • Are we using these in smart ways?
    • How good are we at exploiting gathered information on the next iteration?
  •  Analyse this frequent feedback and adapt processes to correct and improve

Agile methods are very feature-driven – but according to Tom this doesn’t help us work in terms of business goals – so there is a problem in deciding how to maximize the value to business. The key solution to this is to quantify (business, technical, organisational) values.

Tom suggested that the main take-away from his talk was the use of impact estimation tables to measure value delivery / costs. These give a clear understanding of options. (There’s more on these in his book “Competitive Engineering: A Handbook For Systems Engineering, Requirements Engineering, and Software Engineering Using Planguage”).

Using Evo concepts – budgets, goals, design ideas, impact evaluation, requirements spec. with Planguage, estimation, planning & tracking estimation to wrap Scrum concepts gives a complete environment for delivering value to stakeholders – NB just not the customer. Agile methods are too-focussed on the customer. (Medium-sized projects have 40 stakeholders – 50% internal, 50% external). They are stakeholders because they have requirements and need value to be met. (Another take away point– are you identifying all important stakeholders in your projects?) Someone has to analyse values and needs of all stakeholders. If critical stakeholders are denied what they want they can probably destroy project.

Systems are all about building potential to realise stakeholder value – if a system isn’t used then no value can be generated.

[As an aside, Tom took a straw poll of use of agile methods in the audience - Scrum was much more popular than XP – suggesting that XP is on the way out?]

Tom described 10 principles to help get business value from projects.

1. Critical stakeholders determine the value.
2. Values can and must be quantified. (Useful to get to actual values e.g. robustness that can be agreed. Use as clear targets for architecture, design etc.)
3. Values are supported by Value Architecture. (Architecture design is an optimisation problem – most architectural decisions impact multiple values.)
4. Value levels are determined by timing, architecture effect and resources.
5. Required value levels can differ for different scopes (where, who). – (Different stakeholders have different needs, these also change over time.)
6. Value can be delivered early. (Intentionally target the highest priority stakeholders and their highest priority value area – deliver them value early and continuously.)
7. Value can be locked in incrementally. (Give the system to real users – they won’t give it back if they benefit.)
8. New values can be discovered. (Expect to discover new stakeholders and new stakeholder values. Developers must be in dialog with stakeholders / users. This partly arises because stakeholders will come to believe that you can help.)
9. Values can be evaluated as a function. (Tracking stories and burn rates is the only feedback you get in Agile methods. But productivity is defined as reaching goal levels of organisation.)
10. Value delivery will attract resources. (If you are good at delivering value you can expect more funding. Managers like to be credited with success.)

Higher adaptability in a system / organisation is an investment viewed as something that returns value over a longer-term period – Tom argued that this long-term view totally absent from agile. There are however some responsible corporations e.g. HP – in that in those environments you can do this sort of long-term thing without too much debate.

Often nobody has the responsibility that the value be realised – Tom proposes a Chief Value Officer to have this responsibility. But unfortunately didn’t have enough time to go into this in more detail…

The slides for Tom’s talk are available from his web site at:

http://www.gilb.com/community/tiki-download_file.php?fileId=169

CG2008 site visitors - where do they come from?

Blogged under Code Generation, Software by Mark Dalgarno on Tuesday 8 April 2008 at 11:01 am

Regular readers will remember that last year I posted a map showing where visitors to the Code Generation 2007 web site came from.

I thought it would be interesting to see what was happening so far to this year’s Code Generation 2008 site.

The results below:
Visitors to the CG2008 web site. Click for a more complete image.

MetaCase release new version of MetaEdit+

Blogged under Code Generation, Software by Mark Dalgarno on Monday 7 April 2008 at 10:03 pm

MetaCase have announced a new release of their Domain-Specific Modelling toolset MetaEdit+ with lots of new features.

See http://www.codegeneration.net/conference/pressreleases/metacase1.pdf
for the full announcement.

MetaCase are one of the sponsors of our Code Generation 2008 event.

Proudly powered by Wordpress - Theme Triplets Identification Band, the girlish style by neuro