Smart Cities: My Reflections

December 9, 2014

Well as ambigous as this term can get and all encompassing it is, I will attempt to bring forth my thoughts on this, drawing on my two fields of expertise spread over a decade each: Civil Architect and now an IT Architect.

We are in the midst of the change across the world driven by global economy as well as the last few decades of technological progress now converging on internet driven mass consumer systems, wireless or mobile device applications, inexpensive smaller devices and easily programmable embedded devices etc. . Smart City as a concept is evolving from these forces. Many large companies have spent nearly the last 7+ years on various technological aspects of Smart City including Intel, IBM, Cisco and others by their nature of engagement of various roles in urban/ governance management systems they contribute to developing.

Rio-de-Janerio is already touted as the example of IBM enabled Smart City with its central monitoring system and various implementation details. There are few more cities now also making their forays into implementing various systems. Barcelona is creating an environment in which companies can try new ideas and technologies within the urban realm. We in India also have our own efforts in various cities, need to bring them upto global standards and more comprehensive enablement.

First and foremost atleast in developing nation like India, the context of urban planning and architectural development has to be set right. The complete lack of urban architectural controls in most cities is just amazing reflection of lack of that sensibility which was possessed by our forefathers as in a Harrappan settlement, to any in the ancient times. Urban spaces are growing and controlled only by a regime of commercial and political interests, the minimal support for civic amenities and urban planning and its implementation barely done in earnest. That indeed has to be corrected and if “Smart City” with its rhetoric of “Smart” , “Innovation” , “Technology”, “Efficient Monitoring..?” etc. can enable that will be one great goal too for pushing the concept of the smart city, not just for the technological aspect.

Very relevant here is the “Apple” magic; the first wave of interest in Smart City aka Smart Planet has waned considerably. But I predict the second stable wave is set to take off across the world and that will provide for lot of activity that as an UK study indicates will be about 400 billion USD annually, and that is a conservative estimate with known sources of urban activities. Apple’s fortune almost always is built on the second wave innovation, never the ground breaking “never done before” works.

Smart Cities, term that has seen an awakening in Indian Context with the Union Government’s action plan to implement it wider and now with greater political will, than before. I will list some links, I see of interest, at the end of this post.

For most part of the concept of Smart City, the usual constructs of urban planning will continue to be the major enabler and the contraint, with the new age technology modifying, tweaking every aspect of that process and end deliverables, that may eventually result in a visible bigger shift.

I. Overview:

Smart City embraces fundamentally:

Organically growing, sustainable urban area that is enabled with new age technology in socially acceptable manner

Technology should be the enabler and not the pervasive factor to ensure a healthy socially sustainable living in the urban space.

The change should reflect in better city space across transportation means and infrastructure, facilities for hospitals, schools, market spaces, social spaces, as also a proper distribution of civic amenities of electricity, water, waste disposal,  effective policing and judicial systems.

A large list of conceptual areas are involved; some I enumerate below:

* Political: Obvious need for political affirmation and thereby inclusion of all sections of the society. Political mandate facilitates the processes and legal frameworks as well as governance mechanisms. Most technology applications are disruptive and potentially will be undone through irrational arguments if the accepted norms of rolling out change is not adopted.

* Governance, legal frameworks : Effective governance structure with relevant and new rules for a conceptually different urban space is essential. Legal enforcement to limit distortion due to technological change and organic growth is essential to sustainability.

* Urban Plan: Remote sensing based GIS system is typically used today for master plan creation, push for greater use of technology to enable better long term plans and monitoring thereof to adhere to initial plans. This can certainly gain a lot from increased use of specialized technologies now available easily.

* Transport Network: The backbone of any urban space is conceptualizing the pragmatic, capable network of transport that supports as much as possible, mass mode of transports for all residents. Limit individual means of transport to the least level as possible. Envisage and make provisions pragmatically feasible for future modes in say 30 – 50 years ahead. Interconnecting through web, mobile enabled tech of the various modes of mass transport to serve the urban population properly is important.

* Urban Health: Urban – rural divide always conjures up an image of healthy, open spaces of rural settings vis-a-vis the polluted, dense urban population that is inimical to healthier living. Core urban planning has to be fixed to address probably the 90% of this issue, the additional part can be thought through with technology including tele-medicine in every zone/ locality etc.

* Environmental Sustainability: The major thrust in current age is to find all solutions to keep the urban growth that is environment friendly. Go Solar, handle waste locally, limit waste creation, contain resource usage viz: water, electricity to the minimum levels, limit pollution to all forms of environment air, water, noise and earth through housing, industrial or business activities. Allow adequate green lung spaces across the urban space to be sustained.

* Sociological Construct:  The Smart City at its core is for the humans that should have balance to their living and quality of life improved with more use of technology not the other way around. Traditional urban planning theories should hold good and continued with, though with changing social mores, adaptation  be worked in to the plans and policies.

* Economic Sustainability: Technology should enable more economic activity in the city for the citizens and the local government. For the government it generates the required revenue to sustain the investments in modernizing civic amenities and provisioning long term technology assets, renewal on periodic basis etc.

[ Many others not enumerated herein … ]

Lastly:

* New Age Technology:  The enabler, to bring in better control, lower long term costs, assist just in time provisioning, provide real time alerts and controls for all forms of urban management.

Caveat:

Yes technology can enable a raft of possibilities for the SMART City, but need to first align it with the greater scheme of organic growth, the sociological impacts of catering to myriad sections of society, various urban activities etc.

 

II. Technology:

SMART City Technologies:

Given the caveat, let us set out and understand all the elements of technology specially over the backbone of an internet / Metropolitan area network ( Wifi / Wired as it may be). Most of these are already present and working albeit in fractured ways. Many new applications of technology or technology capabilities will evolve in the next few decades. Need to make them work and extend the technology ambit to include better coverage, sustainable long term without breakdowns, redundancy as in today’s systems.

Enable, promote as much citizen, business participation to evolve applications, systems to support adoption and growth of the Smart City technology.

* E-Governance Systems: The most essential enabler to get least effort is expended by residents, businesses, governance officials in ensuring the city works transparently and effectively.

* GIS Systems: All forms of remote sensing images that enable more capable/ actionable GIS based planning and execution of civic works. Monitoring of the organic growth and incremental changes to urban plan can also be supported.

* Transport Network / Traffic Management: Before tech is envisaged, the fundamental concept of mass transport has to be deployed, make individual transport mostly redundant or legally highly limited. Then bring in the technology with proper traffic management system, vehicular systems as in interconnected devices for all buses, metro trains, possibly fire trucks, ambulances etc. to be able to project their information in a manner actionable real time.

* Amenities Monitored with Sensors:  Enable more micro level monitoring for water, gas, electricity usage at locality / large apartment levels for proper distribution, prevent leakages enhance delivery capacities etc.

* Security Systems: Street cameras are used in bits and pieces, a tokenism in traffic management is visible in Indian context, need to tag them to an actionable IT enabled system that will proactively reduce incidences rather than just act as post – facto evidences, or static images that give nothing actionable per se. Non intrusive technology for security monitoring is a small challenge both for hardware and software because of the scale and privacy concerns involved.

* Locality level technology usage: Promote use of solar panels in nearly every building ( if the city is in a solar zone ), fibre optic / wifi enabled apartments, office buildings and other technologies relevant. They can then have automated reports generated for sharing with civic authorities for billing on electricity, water or other charges. Integrate all cables including digital TV, telephone network, broadband etc through a common backbone that rides through the city.

* Industrial and Commercial Space Monitoring: Mandatory sensor based monitoring of all forms of activities that needs control over environmental pollution should be in place, with exclusion list of types of industry not permitted within the Smart City. The monitoring for environmental pollution can also be done throughout urban spaces including office spaces, large apartment blocks, localities if they can have centralized monitoring.

* Technology Enabled Urban Activities: Web-app, GPS, Mobile or wireless technologies are disruptive and changing the face of the world as we know it. The issue for Smart City is to ensure there is necessary process, approvals that will guarantee continuity and alignment with long term objectives. E-Commerce now is fast accelerating to probably create hollow shells out of commercial spaces, web-app integrators for taxi services have spun way beyond governance norms. Instead of throwing the baby out with the bath water, there has to be a planned approach to integrating technology based urban activities with the urban growth.

* Sustainable E-Commerce vs Commercial Spaces: E commerce is to be regulated and supported by commercial spaces that give the right socially sustainable mix.

.. will include more as I edit this further..

Footnote:

The recent Uber fiasco as it is unfolding is a clear example of deploying technology circumventing regulatory, political approvals. The need for planned albeit a slightly slower approach to enabling a smart city.

I would say the immediate reaction of banning Uber across many countries is counter productive to the concept of technology enabling urban economic activity. The web app or technology enabled aggregator concept is a great practical solution, but the political heads should push for governance mechanism around it to make it ideal a solution, though I would still recommend minimal personal modes of transport to be permitted in a smart city.

 

 References:

[1]  http://en.wikipedia.org/wiki/Smart_city

[2]  http://cities.media.mit.edu/

[3] http://www.smart-cities.eu/

[4] http://www.smartcities.info/

[5] http://www.smartcityexpo.com/congress

[6] http://amsterdamsmartcity.com/

[7] http://www.ibm.com/smarterplanet/us/en/smarter_cities/overview/

[8] http://smartcitiescouncil.com/

[9] http://www.citylab.com/tech/2014/01/rise-and-fall-and-eventual-rise-again-smart-city/8081/

[10] http://www.archdaily.com/576480/rem-koolhaas-asks-are-smart-cities-condemned-to-be-stupid/

[11] http://www.enel.com/en-GB/doc/enel_foundation/library/papers/ef_wp5.2014_euricur_rio.pdf

 


Reviving Pharo Morphic View

September 1, 2014

I repost, the updated core package for Pharo Morphic View. From the early Feb 2012 now refactored a bit for Pharo 3.0

http://www.squeaksource.com/PharoGoodies/PharoMorphicView-Core-App-Skrish.11.mcz

Simple framework primarily hooked on with 3 classes and some minimal support classes. ( Most support classes are add ons for more complex capabilities )

To get started:

Browse to:  AbstractSimpleImplicitMorphicView  >> #test .

Run this in workspace:

| aView  informDialogBlock |
aView := AbstractSimpleImplicitMorphicView new windowTitle: ‘Test Morphic View’ ;  windowType: #Dialog . “Try also:  #Window and #Morph”

   “Map widgetName to its morphClass”
aView createMorphsSpecs: {
‘label1′ -> #LabelMorph.
‘text1′ -> #PluggableTextMorph.
‘button1′ -> #PluggableButtonMorph.
}.

informDialogBlock := [ UIManager default inform: ‘Welcome to MorphicView Framework’ ].

“Map widgetName to creation method for model / handler etc..”
aView morphsPrimaryPropertiesSpecs: {
‘button1′ -> ( #on:getState:action:label:menu: -> { informDialogBlock . nil . #value . nil. nil } ).
}.

“Map widgetName to other properties / method to arguments”
aView morphsSecondaryPropertiesSpecs: {
‘label1′ -> { #contents: -> { ‘Have Fun With Morphs:’ }. #vResizing: -> { #rigid }. #extent: -> { 400@30}.   }.
‘button1′ -> { #hResizing: -> { #rigid }. #vResizing: -> { #rigid }. #extent: -> { 400@30}.  #label: -> { ‘Button B’ } }.
‘text1′ -> {  #setText: -> {  ‘\\    Write out a short essay below:..! ‘ withCRs } }.
}.
aView open.

*********************

Now browse through the Tests / Examples for Specs based construction with subclassing either:

* AbstractMorphicView( abstract class for view composition. )

OR

* AbstractSimpleMorphicView  ( simplified api  to provide for TableLayout aka a flow layout…)

implement the above message calls as unary methods in the subclass, with the argument from above as the return values..

#createMorphsSpecs          #morphsPrimaryPropertiesSpec            #morphsSecondaryPropertiesSpec

There are additional API/ initialize routines to leverage in the subclassed construction to make it lot more flexible and scalable.

Potentially there is no limits in components based construction of the UI, with all smaller UIs composing any nth level of complex UI

 

Lot more details are available with pdf / other information posted in early 2012 when this was developed and remained in hibernation.

A full scale MVC mode application can be developed with this framework. The advanced part is to follow through with refactoring for Pharo 3.0 due..

 

PS: Tests works fine except for LanguageTranslator.. which needs a minor refactoring, I will work through on shortly.

Will rethink on the framework though a fresh pair of eyes to make it easier, capable, scalable and easily debuggable as well as make it amenable for a simple drag n drop UI builder. The other target is if we can do a web app UI construction with the same spec..? Seems unlikely in short time but need to think.

 

 

 

 


Teapot: Pharo web REST framework, it ain’t micro

August 28, 2014

Diving mostly into Java these days, it was just sheer bliss to attempt to cook up a sample app with a new framework in Pharo: Teapot. Extend it do a flexible simple app workout.

http://smalltalkhub.com/#!/~zeroflag/Teapot

It is really elegant, what REST does and what Pharo can do with REST, that Teapot amplifies. Really elegant in its code too, the framework brings forth all that is cool about dynamic programming in Smalltalk

The link above  leads one through a 5 min intro / dive into a working set of examples of the framework. I delved for a couple of hours to create a more dynamic example classes that leads me through:

The MCZ:  Teapot-Examples

* Just subclass TeapotRequestHandler and you have now a GET/ PUT/DELETE default for the new class with its appName as the URL path viz:

      http://localhost:8080/NewSubClassAppName   

     ( The ‘NewSubClassAppName’ is default if it is the sub class name given..else assign a name to #appName in the initialize method )

* Override the #forwardTokenA: request method and you are quickly off the block to custom respond to the request. There on, you can follow the example in TeapotLibrary to forward additional tokens of the url to make it return data as reqd.

* The beauty is that in Pharo, now you can change anything, the appName / the response data / any or all of  the URL paths et al in the methods of the new subclass.

* The tests from the browser or using ZnEasy ( as in #sampleGetRequest: ) is instantaneous. None of the edit, compile, restart webserver stuff here..

* This also provides flexibility in scaling dynamically to as many landing URL paths as needed with just a simple discovery of subclasses runtime

From the drudgery of eclipse and its stack to Pharo and its frameworks, Zn / Teapot et als, its sheer bliss.

To try out the examples:

* Load the mcz  and  in a workspace execute:

Teapot stopAll .
teapot := Teapot configure: {
                                     #defaultOutput -> #json. #port -> 8080. #debugMode -> true }.
teapotEx2 := TeapotRequestHandler new initialize.
teapotEx2 runApp: teapot.

*  For quick test run the samples in TeapotLibrary and TeapotSync >> #sampleGetRequest:   highlight and execute the comment lines

*  The inspector that pops up, shows the various test urls configured, and its 200 success code  return object. The urls can be tried out in the web browser  viz:

http://localhost:8080/Library/books/1  http://localhost:8080/Library/search/author/skr  ; http://localhost/SyndLoans/FAC/12HJ3456 etc..

 

The cool dark theme in Pharo is nice on the eye with its contrast too: bit of Indian Flag look with text in white, orange and green along with the soft grey-blue.. nice on the eye.. yellow highlight on the edge.. !

Teapot-Examples01

PS: The Debugger though is cause for concern, it does not align at all with the line highlighted, in Pharo3.0


Nashorn Bug..

June 18, 2014

 

Excited as anyone would be of a newer, performant, cleaner engine Nashorn was something to look forward. Nice to play with moving on from Rhino. But one bug that is annoying and would love to get a fix for fast is its handling of large hierarchy of classes / or large number of methods in the Java class a script calls into. A hack works.. but need better way forward / affirmation on this..

“Step two translates the AST/IR into byte code via the ASM library. ASM emits the actual byte codes to form a script class”

load(‘nashorn:mozilla_compat.js’

 importPackage(my.own.package.name) 

          new MyClass().invokeAnyMethod()  // method on its super class

In this the MySuperClass has invokeAnyMethod() defined viz: MySuperClass should be having a hierarchy of classes that put together have say 6000+ methods.  (The #invokeAnyMethod() then has statements that invokes a method(s) on its super classes, a simple method that returns “hello world” or simple calc on its immediate attributes does not suffice )

In the

JavaAdapterBytecodeGenerator.<init> ( constructor ) .. it does the following:

     gatherMethods(superClass); // this reaps all the methods of the super class MySuperClass of 6000+ methods..

     gatherMethods(interfaces);

..

     generateConstructors(); which in turn invokes the following:

 

private void generateOverridingConstructor(final Constructor<?> ctor, final boolean fromFunction) {

… //this has a loop through all the methods gathered..

for (final MethodInfo mi : methodInfos) {

… which goes on to emit the code bytecodes for the firstMethod of the ClassWriter ( cw ) property

with iteration over 6000+ methods it easily exceeds the 64K bytecode limit of JDK VM method byte code size..

which then gloriously fails as:

 

java.lang.RuntimeException: Method code too large!

            at jdk.internal.org.objectweb.asm.MethodWriter.getSize(MethodWriter.java:2065)

           at jdk.internal.org.objectweb.asm.ClassWriter.toByteArray(ClassWriter.java:837)

           at jdk.nashorn.internal.runtime.linker.JavaAdapterBytecodeGenerator.createAdapterClassLoader(JavaAdapterBytecodeGenerator.java:264)

*******  Hack in at :

public static TypeBasedGuardingDynamicLinker getLinkerForClass(Class<?> clazz) {

   return linkers.get(clazz); // need to check on why NashornBeansLinker for specific cases..

}

// If it returns a NashornBeansLinker that then goes through the chain to dump the ClassWrter creates an issue, for simple methods it returns a BeanLinker and runs fine..  maybe nice to get a hang of it here..

private static GuardedInvocation getSamTypeConverter(final Class<?> sourceType, final Class<?> targetType) throws Exception {

// If source type is more generic than ScriptFunction class, we’ll need to use a guard

final boolean isSourceTypeGeneric = sourceType.isAssignableFrom(ScriptFunction.class); // make this false for such cases… and it works fine.. for the basic tests..

 

 * I have not as yet plumbed as much of the Nashorn logic in this.. why dig through and create a ClassWriter byte code cache (NashornAdaptor to the super class ) when it can probably make a proxy for the Java object and simply let the message be invoked as normal Java method and marshal the result back as per Nashorn semantics.. I am sure there are good reasons, need to plumb it thoroughly before I comprehend it..

*  Now the other concern is org.objectweb.asm.** is used in other packages viz: Groovy / Spring.. et als as asm.xxx.jar  now will this create a conflict is also to be plumbed and ensure classpath loaders honor the segregation properly Groovy still uses a very old asm 4.0 jar.

 

 

 


Pharo is a Smalltalk dialect

April 29, 2014

 

“Invent the future” a resonating statement that we can only be inspired by. There are thousands of more ideas, works that continually inspires us all. Inspiration drives the creation of something new.  Pharo Smalltalk begins with something that existed, but is a new avatar and will probably head to be the best dialect of Smalltalk.

Pharo Smalltalk has begun to establish roots in the minds of so many more engineers/ developers and become more relevant for its use in enterprise application space. The inspiration for its changes from the base of Smalltalk emanate and will continue to be influenced from various sources. Smalltalk is not defined rigorously to be the one published and controlled by one corporation but by the tenets that have held sway and inspired a Java, Ruby and others in their development.

* Image based environment

* IDE, inspectors, debuggers and many other tools as part of its development environment

* Dynamic typed, garbage controlled

* Primarily single inheritance structure

* VM that is rooted in the Blue book, but evolved over time and will continue to evolve even further

* Meta programming capabilities

Many other features add unto the perception of Smalltalk in our mind, not the differences in:

* Morphic UI vs MVC UI or any others

* Platform independent or only single OS focussed ( viz: Dolphin / Smalltalk MT et als.. )

* Ideas and concepts viz: spoon, traits, scratch et als

* Git / SVN filtree rather than MC backed SCM

…… or any other minor changes…

Each dialect will spawn a host of new ideas and will enrich the Smalltalk community. Pharo Smalltalk does currently give a great thrust in that direction and it will be for the greater good to continue to do so say for next few years more before there is a feel that it has diverged far out of its Smalltalk origins to be labelled anything but Smalltalk in its heart and soul, wrapped as it may be that differently

The end goal should not drive an image campaign, the product should be built and shaped to be perfect and the ultimate tool for its use. People as they adopt will shape its image and perception to their benefit and its growth.

Like Steve Jobs,  focus in doing the best with the platform, contributing the best to the platform in whatever way it enables enterprise to adopt and use it forever with guarantees of:

* VM being absolutely rock stable, perfect in its execution of all operations

* Growing body of tools, capabilities, libraries et als

* Interoperability with what exists in simple and fail safe manner.

* Newer exciting projects that it spawns inspires others to try their works on it..

Much more that is already been done over the past 4 years and is expected ahead…

Relabelling Pharo Smalltalk to anything but as a dialect of Smalltalk, I believe is nothing short of denying its own origins for now. Let it mature and diverge enough to be relabelled.

 

******************************

Inspired by a blog from a much admirable Smalltalker:

http://www.tudorgirba.com/blog/pharo-is-pharo

 

 

 


Interesting Tech Reading List

April 25, 2014

Databases: NewSQL vs NoSQL vs OldSQL

* http://cacm.acm.org/blogs/blog-cacm/109710-new-sql-an-alternative-to-nosql-and-old-sql-for-new-oltp-apps/fulltext

including the comments

http://hstore.cs.brown.edu/papers/hstore-endofera.pdf

good start to concurrency is here:

* http://www.agiledata.org/essays/concurrencyControl.html

* http://en.wikipedia.org/wiki/Concurrency_control

* http://en.wikipedia.org/wiki/Multiversion_concurrency_control

Typically I would read through Caching and Transactions in conjunction with concurrency control to relook at an architecture / design of a system.

 Java EE 7 :

DZone RefCardz:

❱❱ API for WebSocket
❱❱ Batch Applications
❱❱ API for JSON Processing
❱❱ Concurrency Utilities
❱❱ JMS 2.0
❱❱ JSF 2.2…

Java 8:

* http://www.oracle.com/technetwork/java/javase/8-whats-new-2157071.html

Compatability Issues for Java 7 to Java 8 migration:

* http://www.oracle.com/technetwork/java/javase/8-compatibility-guide-2156366.html#A999387

 

TDD And Unit Tests:

* http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html

* http://www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf

specially the summary resonates

* http://blog.shinetech.com/2012/06/23/unit-testing-with-groovy/

specially MockFor and “as Class” class instance generation…

 

 

 

 


Pharo 2.0 Released

March 18, 2013

Post your comments…

https://news.ycombinator.com/item?id=5393553

********************************

On Mon, Mar 18, 2013 at 3:42 PM, Marcus Denker <marcus.denker@inria.fr> wrote:

We are proud to announce the release of Pharo 2.0!

You can find information about Pharo at: http://www.pharo-project.org

About this release
——————
All in all, there were over 1600 issues treated in the issue tracker
and 1350 improvements integrated into 2.0.

Major framework changes too:
UI

- Spec
– Widget enhancements
– Layout improvements/cleanups
– Keybindings
– New icons (famfam)
– Growl style notifications
– Revamp progress bar

Developer tools
—————

- Nautilus browser
– Critics browser
– Improved version diff browsing
– Spotlight
– Revamp Code Completion and smart chars
– Interactive navigation using ctrl/cmd+click over classes/methods
– Shout themes
– Andreas profiler

Networking
———–

- Update Zinc
– Zodiac (SSL)

System
——-

- System Announcer
– RPackage replacing PackageInfo
– Command line tools / Headless mode
– Native boost
– Update Ring metamodel
– Fuel serializer
– Freetype fonts

Kernel
——

- DateAndTime refactoring
– Updated FileSystem and replaced FileDirectory

VM

- Latests cog builds
– SSLPlugin
– FilePlugin enhancements
– SocketPlugin fixes
– Included libraries: freetype2, cairo

Cleanups
————-

- FileDirectory removed (replaced by FileSystem)
– SmartRefStream and ReferenceStream removed (replaced by Fuel)
– PackageInfo deprecated (replaced by RPackage)

Other
——-

- Zeroconf scripts
– Continuous Integration for every aspect of our release process.

********************************


Follow

Get every new post delivered to your Inbox.