Lightweight java and .net architecture

Lightweight java means java without entity ejb (if need 2pc, we need a “generic” session bean).


Why lightweight Java? – One reason is that ejb is not easy to test. A better reason: ejb is not easy to develop, and, it is not necessary.

To replace entity beans, we need OR mapping (the most popular is hibernate), or, JDBC wrapper, in a less “advanced” situation. Even we use JDBC, there is no “dataset” – always “custom class”.

To replace specific business logic session beans, we need façade design pattern with “business delegates” pattern. Java has build-in dynamic proxy, so, this is easy – it is so easy that it can be done in-house.

Regardless ejb or lightweight, Log4j, or the later (1.4 and later) build-in logging, is always used for logging.

By doing the above, there is a clear separation between UI and business logic, so, it does not matter what UI it will use, Swing or Web. There are some UI level frameworks; they are very useful; however, no need to list them here.

Further, the business logic part can be easily unit tested without using mock objects.

You may say that, no mock object means it is low-tech. To properly "release" the high-tech addiction energy (we all are guilty of it; admit it; we are programmers -- it is our duty to be addictive to cutting edge technologies– we just need to find proper ways to “release” it), I strongly recommend you use cglib. For property level “dynamic proxy”, which also means “AOP”, we need cglib. You may ask why we need property level “dynamic proxy”. The answer: for a lot of things, for example, property level logging, property level security, etc. – sky is the limit. Anyway, at least it is a better excuse than "mock objects" to get into the same cutting edge area.

By the way, in .net, cglib corresponds to Castle’s “dynamic proxy”. Also, in .net, there is no build-in high level “dynamic proxy”, so, in .net, we have to use Castle’s dynamic proxy, even for high level (“façade level”) dynamic proxies.

Since we are in the neighborhood of talking about the high-tech addiction, I want to emphasize that lightweight java does not mean low-tech java: (a) it is still J2EE, because technically J2EE includes servlet-based technologies; (b) it still uses session beans ("generic" ones); (c) it can use cglib, aop. Lightweight java is high-tech, and is cool :-)

OK, enough java. I talk about java, because I want to find another way to talk about .net.

I have been in both java world and M$ world for a while; so, I want to point out two important history items. Apparently, it is NOT known by many people, even in Java world. However, it is still extremely relevant in .net -- history tends to repeat itself.

The first one is about stored procedures. Once upon a time, it was "also” extremely popular in java world. I use “also”, because I certainly know it is extremely popular in VB6 and even now in .net. However, after entity bean and OR mapping, stored procedures disappear.

The second one is about using command pattern as the foundation for distributed computing. In Java world, it was "also" extremely popular, before J2EE and in the early days of J2EE. I use “also”, because I know it is the foundation for CSLA, which is a very popular framework in .net. Before J2EE, IBM had a very ambitious project called “San Francisco project”, it was based on command pattern. In the first a few versions of WebSphere (the flagship app server from IBM), command design pattern was prominent; however, it did not last.

Perhaps .net will not repeat the same route; you never know; no crystal balls; let’s wait and see. Regardless, knowing Java’s history certainly makes it more interesting.

Strongly typed collection is evil, general collection with casting is ugly, generics ends both

vikas has a nice post: Generics – The Code Killer
http://vikasnetdev.blogspot.com/2006/06/generics-code-killer.html

The following is my comment (double post to here)

"Performance" and "type safety" – to be frank, I do not really care either :-) -- the "performance" can never be the hotspot in a database applications; the "type safety" is also not a real problem – who will put a chicken in a fox cage anyway :-) – the really thing is that it removes the ugliness. Whenever I have an argument on whether we use typed collection or casting, I always say casting, but I am always embarrassed by the ugliness.

You are totally right, it comes from classic VB; and the casting is from pre1.5 (“classic”) java. Generics puts the end of both.

I also want to add that it is not just an issue of saving number of classes. It is also an OO concept thing. The reason we say that strongly typed collection is evil (i.e. totally wrong) and casting is just ugly (i.e., bad, but OK), is that strongly typed collection uses an OO model that is obsolete for quite a while (mid-80). Through the experiments in C++ and Java from mid-80 to late-90 of last century, we know that “top level collections” is a bad idea --it should be discarded to a corner (you use it as a last resort). Classic VB was never a real OO; therefore, it had very limited OO experiment community; as a result, it kept the obsolete OO concept, and passed it to .net community.

In "modern" OO model, there is no top level collections, only classes, and 1-m, m-1, m-m class relationships.

8 Core OTTEC Techniques v2

This blog (I added "8" in the title to make it easier to remember the key points) is effectively my “default” architecture: it does not mean that I will blindly use it everywhere; it means it is the starting point of my thinking -- software is soft; it is about the tradeoffs. However, we do need a starting point; we cannot think everything from basic OO or language syntactical features.

As a result, this blog is likely to be re-posted regularly, to reflect the changes of my architecture thinking.

----------------------------------
After “8 OTTEC (123 Enterprise Computing) Issues” and “Core .net Tools and Frameworks”, here are the “Core OTTEC Techniques”.

Note that I keep the technique item numbers corresponding to the 8 issues; also, you can find the links to the tools and frameworks in the previous post, or, in the “Links” section of this blog site.

OTTEC Techniques:

1. Use log4net, in a simplistic style (avoid fancy stuff)
2. Use MS Ent Lib’s data access block
2’. Use Nhibernate, instead of Ent Lib, when we are in “advanced” situations


3-4-5-6. Use the DynamicProxy in castle framework to centralize remoting, transaction handling, security, and logging, at the façade level.

Note: In order to do that, Use programmatic IoC (i.e., centralize all “Factories”); but for most projects, do not use XML style IoC (that is too heavy)

--- to avoid client casting, use factoryMethod
--- use abstrctFactory for switching easily
--- put class name, method name, variable name in the factoryMethod



7. The above security and logging can be at property level;
7'. Use the custom class business rule validation technique in CSLA; but via AOP (see above about AOP, but at property level).
7''. Use the “custom class databinding” technique in CSLA; but via AOP (see above about AOP, but at property level).
--- class-to-form using custom event
--- form-to-class (onchange; no exception in set; use rule)

--- tip: listview/read-only-grid read-only: this is good for user anyway;
so, no need of typed collection (note: sorting etc are control's business)
--- tip: update's return object: only get the seq and update the property
do not try to replace the whole object
--- tip: use read-only properties to "borrow" fields for m-m-like
--- tip: inheritance or parent -- pass reader, tx, parent etc.


--- "CSLA with dynamic proxy/aop":
(a) not limiting to 4 (CRUD) methods;
(b) so, no need of special workflow objects,
(c) so, no need for special “criteria” and “exists” classes.

(d) At property level, cleaner property/get/set,
because no CanRead/WriteProperty, PropertyHasChanged calls.



---- Note that I said that we use techniques from Java; but CSLA originates from classic VB and VB.net. Further, the “custom class databinding” is indeed a new thing in .net. I admit that it is close to an over generalization; however, please also note that the idea of “custom class and business rule validation” is the mainstream in Java. There is no dataset "pollution" – at least in the days when the mainstream convention was established. The same logic applies to MS Ent lib’s data access block. Wrapping JDBC is a routine, mainstream practice in Java -- that is, if you have not graduated to using OR mapping (e.g., Hibernate).

---- I intentionally use the “over generalization”, is to emphasize that it is a painful learning experience for a classic VB team, to introduce those techniques. The fact that CSLA originates from classic VB, and MS Ent Lib has no direct Java origin, does not change the fact that those things are “routine”, common sense practices in Java, but in a VB team, typically you will have to have serious uphill efforts to introduce them, one by one, sometimes.


8. Use Nunit, but only on façade methods

Use blog as top level notes. Also about email, electronic notes, ISO 9000 and RUP, and tablet PC

Use blog as top level notes. Also about email, electronic notes, ISO 9000 and RUP, and tablet PC


To make sure that I keep writing blog and to keep myself honest in writing, I will stop writing my top-level “notes”, and start to move the content to my blog. Of course, I will still keep my low-level notes, because they contain some of my passwords :-)

Immediately, we need a rule: I change my notes often; however, I should not change my blog once it is published. The trade-off: for small changes, I post comments; for bigger changes, I post another blog, provide link(s) to its previous version.

----------------- More elaborations:

I know that I will need sometime to get used to it: we sometimes put some obviously incorrect, yet creative ideas in our notes. Because the notes are private, so, it does not matter; but now, we need to be careful. However, we still want to keep the creativity.

However, it is doable; because we have done it before. I guess it started with emails. Looking back, using email, as innocuous as it seems, is revolutionary in our technical thinking: by email, we begin to think while we do “persistent” communication (i.e., talking does not count), instead of think first and then communicate. I know it is not that big deal in political thinking ;-), but for technical thinking, that is a novel thing.

Using email also helps us to ease the pain to switch paper notes into electronic notes; more accurately, using emails makes it a necessity to make electronic notes, because otherwise you have to maintain both emails and paper notes. However, because the concept of “(extensive) electronic notes” is so difficult, it takes a while. Often I can still see people, even programmers, using paper notes as their “primary” notes..

Sometimes I feel that the most difficult concept in ISO 9000, UP, RUP etc. is simply the fact that it effectively requires people to take electronic notes. I understand that there is a huge “organizational” element here (“tacit knowledge ownership”). However, it seems that an equally important factor is a simple cognitive one: people are not used to write electronic notes, yet.


Why? Simple: It is still too difficult. Writing electronic notes is still not at it prime time, yet. You may say I am crazy. No, I am not. It is amazing that it took so long to do such a simple thing.

There are a lot of challenges. I do not want to list them. The good news is that its prime time is coming. I bet it is a TabletPC, with voice recognition, handwriting recognition, build-in camera/scanner and voice recorder, wireless Internet and phone, and with processor, memory, and disk space equivalent with its contemporary desktop/laptop PCs. The last one -- although sounds ambiguous -- is important, because we need one (virtual) machine as our primary machine that can basically do everything; otherwise it is will be too confusing. Note that the word “virtual” is important here; perhaps we will use a TabletPC as the human-computer interface, and the TablePC will communicate with other things.

You may say, why TabletPC? Because human-computer interface: we need typing; typing requires certain size. The minimal size is TabletPC. Until we have affordable and reliable electronic paper or/and light projection keyboards/screens, which we may have after at least another 5 years, we have to use TablePC.

Ajax

I am going to work more with AJAX.

The key is, of course, the “A” part ("asynchronous", or you may say the X stands for XmlHttpRequest-like; however, XML is not relevant here, it is almost a misnomer) -- it is asynchronous with the browser’s main page. You may say, we used “frame” for that. Yes, you did, but that was hack; this is the norm, and, of course, more powerful.

Yes, all its elements have been there for a while -- even the "A" part; but they hav not been "integrated" until the concept/term of "AJAX". Perhaps “google” makes all those techniques suddenly popular; howevere, perhaps, the bigger picture is that broadband is really taking off now, and it is time for us to have a “new” Internet programming/usage model.

Now, browsers have the same power as that of the windows. However, browsers have one more responsibility: it must be “backward compatible” with different browsers (“browsers war”), and their previous “page-centric” (or “document-centric”) usage model. “Browser war” can be handled by the similar techniques we used in the past (it is more complex now; but we can get more help from Atlas-like things); however, the trade-off of the “page-centric” model and the “asynchronous” application model is new, and is very interesting.

The key: do not feel that AJAX is a new technology and therefore simply dismiss the page-centric model. There are reasons for page-centric model: (1) web started with static html; (2) users are used to page-centric; (3) you may ask: why web started with static html? – The answer: it is not just web. The most popular productivity tool and the cash cow for the almighty M$ -- M$ 0ffice is also document centric! The whole business of office-automation is document-centric. In addition, another big piece, “report”, is also document-centric. My point: application centric and document/page centric are much deeper than window and web; they are within each of the platforms.

I noticed vikas has at least three blogs/links on this topic. I recomment start with reading his blogs, and of course, google search.
http://vikasnetdev.blogspot.com/2006/04/how-i-stopped-worrying-about-browsers.html
http://vikasnetdev.blogspot.com/2006/04/atlas.html
http://vikasnetdev.blogspot.com/2005/07/ajax.html

total rewrite and the concept of software service

This is a very nice post from vikas: http://vikasnetdev.blogspot.com/2006/06/what-if-we-can-maintain-our-software.html

I thought about it recently also (http://survic.blogspot.com/2006/06/legacy-enhancement-development-why-and_06.html). However, I elaborated the “why” part only from individual developer’s perspective. Your blog reminded me to address the bigger picture.

I want to push your post “a little bit” further.

(1) Rewrite is always much more costly, and more importantly, much more risky (more than half will fail the first try).
(2) With good service, software can live forever -- it can even evolve, individually ;-)
(3) Combine the above two: “total rewrite” is a bad sign; it usually means the “car” is abused.
(4) Users depend on professionals to help them. Educated and informed decision is the norm; however, a lot of times, “let us sneak changes” is also necessary. It means a lot of responsibility, and requires real “fearless leaders”. It is sad that we see too much “total rewrite”, and too little “software service”.

To mock or not to mock

this is posted in http://forums.lhotka.net/forums/2/1384/ShowThread.aspx#1384


To mock or not to mock: We really need to be careful here. We need to put things in perspective. TDD is THE agile and THE lightweight methodology – that is, before .net getting into the picture, i.e., when we are talking only in C++/Java context. Now, .net brings in VB tradition. VB has been in a RAD context, which is much lighter.

Before you dismiss my point lightly, note that I used Java and I love Java. Also, I used TDD and I love TDD. Of course, I also used classic VB, VB.net and C#.

My point: if you really want the lightweight end of TDD, then, do not mock. Mock is for developing frameworks, not for developing everyday applications.

A healthy way to resist the temptation of using mocks in developing everyday applications, your can participate some open source framework developments. In that way, you learn how to mock, the right place, and not mess up your everyday application developments.

Note that once you mock, your unit testing code is totally different from your real code; as a result, your unit testing code loses a key functionality: documentation.

legacy enhancement development: why and how

It has been for a while since my last blog, because I have been doing legacy enhancement development (or, you may call it support-level-3 development). Why not rewriting it -- you ask; well, you cannot rewrite everything all at once, can you?

From the perspective of an individual developer, there is actually an advantage of doing legacy enhancement development. It is that you can cover a much larger amount of code base – you can “own” a subsystem in a short period of time. If we are talking about working in a stable company and therefore it is about a few years time range, then, it is the best way to gather requirements about a subsystem. You may say, I can read the functional spec. My answer is: who writes those functional specifications – often those things are not accurate, not detailed enough, and not updated – if they are available at all. In other words, if you are a good developer, then, the support-level-3 development is making you into a good business analyst and a good architect, simultaneously.

Of course, I realize that some support developers are not motivated to learn new technologies; but that is another issue.

OK, so much for motivation speech. It is the time for the question of how.

I heard about people talking about that to change a legacy system, you should first develop a suite of unit tests; only after that, should you change the system.

It is a good strategy -- if the legacy system is pretty modern. However, if the legacy system is a real “legacy” system, then, very likely it is not cost-effective to do that. You may ask, why? Simple: if the business logic is interwoven with UI, then, it is very costly to develop a suite of unit tests.

Even if developing a suite of unit tests is practical, there are higher level things that need to be considered first. There are three items:

(a) Collect and be diligent about software inventory info. Legacy system tends to be in a status that it does not have basic inventory information. The most fundamental two are: where the source code is located and where the installation kits for third-party packages are located. Other questions can be derived from those two.

(b) Have a high level (“power-user” level) understanding of the system. Legacy system tends to be in a status that it does not have user manuals and any other documentation. So, developers must identify “power-users”, and, communicate with them often and effectively. Often peer developers are also helpful. However, it is important to note that in most cases, peer developers should not be held as the definitive source.

(c) Add systematic yet quick changes to make the system easier to maintain, without “intimate knowledge”. For example, a typical example is to add database access logging to the system. Another example is to identify existing regularities, eliminate “outliers”, and articulate some “coding standards” to make the regularities explicit and enforceable.