please ignore

test See all comments

Other SOC reports

Some other reports from ruby soccers, they seem to agree with my previous analisys .

• Ilmari Heikkinen, GlimR (part 1, part 2 )
• Gregory Brown, ruport (here )
• Jason Morrison, RDT type inference (summary )

If you find others please do notice me.

See all comments

Google Summer of Code: Ex-Post thoughts

Ok, google’s SoC is ended, and I had some spare time to repair this blog which was broken for a while, so now you’re getting some random thoughts on it and on my work.

If you ever plan to try it in the next years you may find this useful, possibly.

Lessons Learned

• get a timeline, set milestones of what you should have done in a week or so, this prevents the bad feeling of not knowing how much you should do tomorrow and the fear of not having a chance to get stuff done in time.
• write tests, and really check that they are working, I lost a lot of time because I was testing against old installed rubygems instead of my work. Life just sucks, sometimes.
• if the main repository is unstable go away, in the first part of my SoC I tried to work with the main repository, which was under heavy refactoring. This meant that basic functionalities got broken and thus I should go around them. Try to find a stable repository, if you have good tests you should be able to port your code to the main code base (quite) easily once it gets stable again.
• stay in touch with other devs, the idea of SoC is to be part of a community, and that is always the best way to find help, answers, and moral support. I owe a beer to everyone in #nitro, possibly.
• KISS & YAGNI, ok, it should be obvious to anyone that these acronyms are important, but well, it is hard to really respect the principles.
• find a good mentor, mine was nice (fining out we both coose to learn AliceML this year was incredible :) but I’ve heard quite a bit of people that lost contact with her/him, and that made developing much harder.
• put out your code soon, set up a personal repository if you don’t want/can mess up with the main one, it makes easy for people/friends/mentors to review your code and get early warnings about it. Being shy about your code is a bad habit. Possibly this extends out into real life.
• get a good development platform, take time to setup it properly, and do investigate small problems about it ASAP, I lost a lot of time due to subtle differences in the DBMS backends for Og because some are unstable due to recent rewrite.
• make small steps, “writing a ui” is an extremely hard thing, “wrap errors in a div” is a 5 minute thing. If you use darcs try record‘ing after each small step.
• facets rocks, search stuff in it before you reinvent the wheel.
• never ever hide errors, if the framework does, change its behaviour as long as you work on it, you can make it safe (or utterly broken, depending on your point of view) again later. If I ever see “rescue Object” again I could start crying. (Luckily those are being completely removed).
• get some demo from the start, the Django tutorial was a great source to start hacking towards a meaningful objective for an admin ui, for example. This allows discovering corners that you may have not thought of in the beginning.
• avoid distractions, and I do not mean real life or eventual Anime video, I mean getting deep into solving completely off-topic issues just because they incidentally relate to some remote part of your work. Having an Array like [1,2,3,5] somewhere in your code does not mean you must rewrite the Prime class and investigate the Riemann Hypothesis.

Results wrt original Plan

• new gen infrastructure [ok]
• gen scaffolding [ok]
• gen administration [ok]
• proto code simplification [ok]
• order and choice of shown fields [ok]
• order and choice of shown classes [ok]
• input methods for complex fields [ko, only relations, need new control system for stuff like tags]
• choice of navigational UI [ok, easily reusable/hackable breadcrumbs & menu helpers]
• AJAXish stuff [ko, needs more work]
• per-part settings [ok]
• various scaffolding settings [ok]
• online configuration [ok]

Various added stuff

• tests
• automagic methods allows usage of default to_s, to_href and to_edit_href with Og managed objects.
• helper to build a table of objects
• helper to show a single object in table view
• helper to show errors in meaningful xhtml
• NavigationHelper
• gen/part allows easy importing of an existing part into a new project
• automagic search key allows searching objects through an attribute by setting a single setting
• yaml export of the dbms
• customizable table links (could have more settings for tables though)
• proto split
• SystemPage element allows easy hacking of the admin view
• stylesheet (please do help, I’m very bad at css design)
• attribute_to_html
• visual feedback on successfull creation and errors with better handling of errors (i.e. data are kept in the form)
• some og and nitro fixes
• merged 30+ patch from devlab repository into main one, plus conflict handling
• some ui reworking, new view.xhtml, search.xhtml, admin/index and other.
• the great two-line header :)

Conclusions

This was uber cool, I wish I had more time back in july, but well, my last exam was something like 27 of that month. Obviously I plan to keep work on the administration part, the code needs more refactoring and I’m thinking of at least three new settings (multiple-attribute search key a-la streamlined, non-tabular view for objects and breadcrumbish title helper), plus I need more tests.

If you want to take a peek at what I’ve don you can look at the temporary video available here , which shows a part of the available features. Please do not consider this a permanent link, I could remove it as soon as I have another video showing more stuff.

Thanks

To google for organizing the Summer of Code, to RubyCentral for choosing me, to Bryan for being my mentor, to George for Nitro and Tom for facets, to everyone in #nitro, #ruby-it, #ruby-lang and #verbamanent for uncountable help and fun.

See all comments

take 2

now if you really see this the migration was successfull :) Sorry for the noise. See all comments

blog problems

if you notice yor comments disapperaed, posts are misordered and basically everythig happens as new in the feed, well, sorry. Some problems upgrading typo, which I'll possiblt try to fix later, now SOC has higher priority :/ See all comments

@template_root evilness

or: why the hell if I start run.rb, access /admin/foo, /admin everything works fine but if I access /admin , /admin/foo then the template root get messed up?

Life just sucks :(

See all comments

More Settings for everyone

So today bI was able to add a setting to handle the appearance of a menu, I put refactored breadcrumbs so that they appear only in the system skin (and another setting for that, I already did this but was broken). Then a setting for the elements in the menu, defaulting to all the scaffolded controllers + AdminController.

Also did some nice xhtml cleanup, removing evil <br> tags, and making the search page more obvious (now you can understand that even if search will not work withouth a Model.search advanced search still will, due to it working via query_by_example).

I also checked (didn’t for a while) that everything is working fine with relations. Answer: no, if you’re using SQLite, it is utterly broken. took me a lot to find out this because, for some strange reaosn, I had other bugs that suddenly disappeared once I colsed my terminal. Go figure. Anyway, I was able to use it with MySQL, so the problem is in Og’s SQLite adapter.

Also, pulling the latest patches from george seem ok, just a tiny microconflict in Og.setup. I can even tell you that facets 1.7 breaks Og horribly.

Then I went to refactor the controllers so that every scaffolded thing inherits from AdminController when they are generated together. But guess what? I impacted in a strange problem: it seem that the template root get messy when using inherited controllers.

Given FooController and a subclass BarController the template path of the former will be used with the latter, obviously causing bugs.

But I’ve been hacking for something like 15 hours so I need some rest.

See all comments

Merging devlab with repo

So I finally was able to run Gen tests correctly on repo.

The patch is here for anyone who wants to take a peek at it, but be careful, it may be a little messy, because some of the conflict solving was done wrongly, and repaired with later patches. The result is good, anyway, and I thik it is really nice that a large part of the work done on making Nitro 0.31 solid can find its way into 0.40.

Now, only 20 patches are left, and not all are important (i.e. release files)

All the Nitro tests that were passing before are still passing, as for Og.. I didn’t look at the patches to it, because almost everything seem to generate a conflict, due to the recent Og/adapter refactoring.

Sadly, my admin interface at the moment is still not working (tests pass but ATM they just test that the generators library is working and creates proper files).

The problem relies basically in two things: usage of @base in the breadcrumb line, which seem easy to solve, and broken (or just different) handling of redirect, which causes errors in many actions.

Anyway, the new nitro code seems nicer than the old one, for example form is way better thean the old messy form_for, and the fact that I was able to reproduce the latter as a 3-line layer over the first is a good sign :)

In the process, I learned a lot about darcs, thanks to the nice people on the freenode #darcs channel and my friend Paolo (which is doing some uber strange things in haskelland :)

The list of my tiny additions to the nitro helpers got a new member: form_errors.

This is a nice tiny method, which uses informations stored in flash[:ERRORS] to create a tiny div filled with error messages. Basically just moved George’s new code for the scaffolding view into an helper.

So, you can have a page like this:

    
    #{form_errors}
    Insert data:
    #{form_for(Person}}
    
    

and the errors’ div will just appear when loading the page with a proper flash set.

Making the test run was quite hard due to many little differences, for example the disappearing of #properties, which means that to discover the attributes of a class you need to use serializable_attributes (needed both in the administration views and in the table_for_objects helper).

This led me to a nice thing: my first usage in real life of the fail assertion, wich I used in the Unit Tests to ensure that #properties is not called anymore.

As usual, please do review my code and the patch if you can, and don’t be afraid of criticizing it, I know I’m not so good ;)

See all comments

Ho to work on devlab and send to repo

Ok, I always had this problem of working on devlab and wanting to send patches to george. Unless he goes around hunting missing patches he could get errors like this

    Cannot apply this patch bundle, since we're missing: 
    * foo bar patch
    

The solution is to bundle any needed patch when you do a send. If you have an internet connection you can do

    
    me@/devlab$ darcs send http://repo.nitroproject.org -o bundle.patch
    
    

(possibly you could do the same with a local copy of the repository)

Warning: this would make your repository work with repo in the future, so you may wish to use a line like this:

    darcs send --no-set-default http://repo.nitroproject.org -o foo.patch
    

See also the entry on making patches to nitro at Oxy (which I hope could get updated with this :).

See all comments

table_for, wasn't it there?

I’m trying to make the scaffolding/administration template as small and easy to understand as possible. At the same time, trying yo make it easier to read for the user, the current “listification” of the objects uses inspect/to_s to show them, which makes really hard to grasp what they are when you’ve not redefined the methods properly. So I was trying to replace the ugly table code with an helper.

And, I was quite sure, there was something to build a table out of a list of objects. It comes out I was wrong, or at least I’m unable to find it again (pleeease do correct me on this if I’m wrong) so I wrote one:

      # Creates a table automatically based on the structure of the 
      # objects passed as argument. Objects have to be non-empty
      # because the headers are built via reflection on the objects.
    
      def objects_table_for(objects, options={})
    
        keys=objects.first.class.properties.keys - [:oid]
        headers=keys+['Actions']
        values=objects.map do |o| 
          ary=[]
          keys.each do |k| 
            ary << o.send(k) 
          end
          ary <<  %~<a href="edit/#{o.oid}">edit</a> <a href="delete/#{o.oid}" onclick="confirm('Are you sure?')">del</a>~
        end
        options= {:alternating_rows=>true,
                  :id=>'objects_table',
                  :values=>values,
                  :headers=>headers}.update(options)
        table(options)
      end
    

Suggestions on the above code and on the name are welcome before I send a patch against devlab and repo.

Anyway, this allows me to reploace 20 lines of view code with one, at the cost of making it less configurable, but making it much more easy to find something.

Two problems still open: it seem that scaffolding does something if the things are Orderable, but I don’t understand what, please provide an hint if you can. Also, Date/Time will be shown in a classical to_s fashion, I’m wondering if I should add an option, if I should yield a block and or use some kind of Setting, suggestions welcome.

Oh, sorry again for not updating the blog, I’m coding nicely behind the scene, just in a floating state, as you can imagine if you take a look at my dev memo (in italian)

See all comments

Campioni del mondo!

sorry, had to that, we are the best team of the world, for the second time in my life.

See all comments

Back to code

Finally I’ve found some more time to code. Actually, talking with Bryan we established a mini timeline to bring up a nice feature for the admin view, the ability to change settings from the web and save them as YAML code for future reuse, but before that I found out I did something really wrong WRT the admin geenaration (plus I found some problems related to reloading settings).

The problem is that I was inadvertently reusing the builtin part/admin instead of the one I generated so my testing was useless.

I’m now recapping the work, and in the process trying to clean up the scaffolded views.

To simplify them I’ll try to add table_for and to remove the strange needed for proper path setup.

At the moment, all the views have something like

     <?r base="#@base/%base%" ?>
    

to allow proper setting of paths to actions.

@base is the base path, i.e. for AdminController it is / and for FooController mounted over admin it is /admin. Ok. base is gsubbed at “compile” time with the real controller name/path. Ok. Lastly, base is overriding a Nitro::Controller#base method wich allows access to the above instance variable. All in all, not very “obvious” code.

So I ended up sending a patch to the ML that adds a Controller#path method to access Controller.mount_path, the code could then be refactored nicely, because what @base/%base% do is basically recreate mount_path.

Talking about refactoring, I was thinking that we could make use of facets’ __DIR__ which allows access to the directory of the current file. It seem there are quite a bit of places where this is approach used in all Nitro/Og, so consolidating them with an already available method seem A Good Thing.

See all comments

Long Time No C

Hi fellow readers, sorry for being away a long time, sadly it is deep exam period here, and I was sidetracked in working on some home projects doing linux (as in kernel) hacking. Given that I know very little C and assembly (“wtf is {.foo=bar} ?”), you may understand it took some time.

Anyway, back to nitro, I’m now working on the next little step which is: allowing modification of Settings from the Admin UI, which I hope is doable easily, even if it does offer some interesting problems (i.e. handling of non basic ruby object as values) which I’ll try to investigate today.

But you may be wondering what happned with gen admin. Well, here it is. Sadly I had this in a seemingly working state for some days but had no time to push it here. I’m not 100% happy with it, and I’m wondering if it would be better to massively remove some of the code, like the stylesheet stuff (let’s just provide it from gen app), and all the reference to @base in the templates, which seem useless to me (which may not mean that they really are).

George if you’re reading please do comment on this (and keep an eye on your mailbox, where I sent you a message about my problems with Elements as you suggested ;).

I’ll try to be a better blogger/coder, sorry again guys/ladies. Oh, and don’t worry, once my exam season is over I’ll have much more time to hack on nitro :)

See all comments

update on admin generation

sorry for being away for some days, I’m quite busy with real life, and someone I knew passed away (not a too close person, but close to a friend).

I’ll be back to hacking soon, a mostly functional gen admin is resting in my box, and tonight I've got a meeting with our local ruby user group where I'll be speaking about my SOC project. Who knows, maybe I can get someone to try out nitro.

See all comments

Gen Refactored !

The fun of refactoring gen finally reached a local maximum with which I mean that I’m satisfied with the code even if not perfect and that it will need a new coding sprint to get past that.

The new code (patch) is very easy to understand: you find a class based on the first argument via Gen.load_generator & Gen.generator_class, then you instantiate it with new call run on it.

The error handling is done via a simple error method inherited in the various Gen subclasses, which allows consistent erro reporting.

I even put in my old idea for better error reporting when using generators, in the old days you’d get Cannot load the specified generator! in this cases:

• wrong generator name
• Syntax errors in the gen.rb file
• failing require@s in the @gen.rb file

and every other LoadError, now a specific error message is supplied.

The only thing I’m not 100% happy with is the dependency of the code on a small naming convention, the classes have to be called SomethingGen where Gen is not optional. This allows to find them dynamically via a simple const_get and is far better imo, than relying on globals, because it allows simpler testing and reuse, but may not be really perfect.

I’m thinking of using ObjectSpace to find each subclass of Gen (facets already has a Class#subclasses method IIRC) but I have not thought much about it, because it seem seamingless complicated. Suggestions are welcome.

Funny Horror Story

Anyway, moving functionalities into lib/gen and out of the generators and the executable script allowed a nice removing of duplicated code, and now we have the nice feature of knowing for sure what are the available generators (in the old script they were hardcoded), but in the process I did encounter nasty bug that took me half an hour to fix.

Basically, the gen script used to load lib/gen.rb and had some code in its body, withouth any method def. The generators, OTOH had a lot of had duplicated require that could be pushed into the superclass.

So the first step of the refactoring was to define methods containing the script code. Check, everything working. Then move them to the Gen class. Check. Then move duplicated require from the generators. Fail.

Sadly,I did not run the tests after each step and the Curse Of The Dead Tester hit me in the face, all the generators were suddenly unable to access files that I supplied them (i.e. gen form model.rb was raising a LoadError) and I could not understand why.

I blamed win32 and Unix and ruby for a while, I did rituals to calm the bad spirits that ate my $LOAD_PATH and wondered where the hell the files generated from gen app were disappearing.

The I discovered that when using Nitro as a library you have to explicitly tell it to not change the working directory by setting the $NITRO_NO_ENVIRONMENT variable before you load the file and not before you use it.

So I could send the bad spirit to rest, and now everything is peaceful again.

See all comments

Quick update: gen cleanup

There is stuff to be done so that everything is wrapped up nicely, but it seem that the dummy tests I added are passing, my smoke tests are working and the code has been reduced. Let me say I like how we can use ruby’s arg number check (read: ArgumentError) instead of manually doing it in all the generators :)

Patch available as usual, but expect more changes before I submit it to nitro ML. Sorry, real life demands time :(

See all comments

cleaning up bin/gen

I started writing gen/administration, leaving the xhtml cleanup/helper creation for later, since I want a little more testing in place before.

But why I want all this testing? If you try to follow my reasoning I’ll try to give an explanation.

What is needed for gen admin

It is quite simple to think of a generated part/admin as something that loads different scaffolded parts for each one of the Og.managed_classes and adds some more stuff like setup, a proper <SystemPage> and so on.

This cleanly translates to something like

    def run
     create_admin_files
     create_parts_for_classes
    end
    
    

where the latter can be done either via creating multiple ScaffoldGen objects and calling their run method or making AdminGen inherit from it and using super multiple times.

The Problem

But how does bin/gen handles this? Well it basically doesn’t, it just loads a specific file for every generator and let it handle the setup directly, thus each generator

• accesses parameters directly via ARGV
• creates a generator and binds it to $generator

This makes the above idea messier to implement cause there would be a time dependency on who sets the global, and on the access to the shared ARGV. Also even if now it is possible to add generator-specific usage messages, the excutable script would not tell you anything about them, cause it relies on a method-local text.

So what?

Well, I need some small simplistic tests for the other generators, so that I can then refactor everything and be sure I have not messed up anything.

And in the process I think I can make the whole thing easier to test for people hoping to add other generators, thus making it easier to contribute. IMO letting third party stuff merge nicely into the framework is the best service we can offer to developers.

See all comments

Quick update: tests in place

Now I have a first draft at unit tests for gen scaffolding, they’re not really checking that everything works, but are ok to check basic file creation, and I feel a little more confident on the working path. Obviously I fixed some bugs wholxC3xB2e writing those, TDD rulez :)

You can get the new patch here .

Which makes me think that I could setup my own darcs repo, and let george/bryan pull the changes they want, I’ll check if my kind hosting provider will let me do that :)

See all comments

A first step for Static Scaffolding

note: this article originally appeared as a preliminary note, sorry. Also I expected to complete the first work in two days but, well, university duties stepped in :/

---

The first step of my work is to let nitro generate static code for scaffolding a class. This is not really hard since the controller can be abstracted away in a simple way.

The approach I’m currently using is to generate a part, which in turn contains a controller, two elements for a form and a set of xhtml files for the views.

The ruby files are loaded from part/Classname.rb (should I downcase that? ) and the loader also takes care of mounting the controller at a default location of /classname.plural.

I played with the idea of generating the controller via ScaffoldingCompiler but it resulted in uber complex code for a little gain, so I settled on using a simple skeleton controller. Obviously, once I had a skeleton controller, I thought that having similar skeletons for the elements and for the loader was the logical thing to do so I did it. As for the views: at the moment this files are just taken from nitro/proto/public/scaffold but I plan to use a different approach.

I wanted to use elements in the building of the scaffolded UI, cause they generate simple view code, they are easily reused, and are easy to combine.

But I always misunderstood Elements. I thought they worked like CGIKit’s components or like a great mix of rails partials + layouts, but the fact that variables are statically bound make them much less useful. Now I think I understand why everyone is using elements just for skin.rb :( (Messing up with the compiler pipeline is something I want to avoid in scaffolded code)

So, for the next next step I’ll generate helpers in addition to the elements. This way the elements will be there as some kind of example code, and I can keep the templates simple using helpers.

I was a bad developer, and have no tests in place as for now, which is Evil, and is a problem I need to fix. But I have a preliminary patch which I’ll be posting to the nitro ML as soon as those are written and I polished the code, if you take a look please let me know what do you think.

See all comments

Summer of Code Project Blogging

Hi everyone, starting now I'll use this blog to keep track of my ongoing work for Google's summer of code. If you're wondering what my project is, here's basically my accepted proposal. The "Nitro":http://www.nitroproject.org web development framework allows the developer to autogenerate a simple administration page to manage all the model objects via web, but the current implementation has some issues: * the interface is only generated at runtime * the administration UI is not easily configurable, and is hardly useful in finished products Since I'm usually lazy, I want to be able to get a working nice admin interface and keep that with some tweakings, but these two points make it hard to do so. So as a first step I'll try to add offline code generation for admin and scaffolding, then, I want to leverage Nitro's settings system to introduce some configurable things. There is huge room for configurability in the admin UI, such as: - the order for fields and managed classes - classes/fields to show and how - what kind of input method to use for complex fields - whether we want to show system informations (cache infos, url mappings) - the choice of the navigational UI for sections of the admin interface - AJAX-ish everywhere or not On the way to do this I'll even try to come up with a minimal auth system. No, it won't have ACLs, it would not be RBAC and in general will be dummy, just users and groups. Well, at least this is what I have in my mind as of now :) Oh, well, in case you're wondering why I'm working on Nitro and using a Typo blog.. well, I do like rails (and I hope I can port this work over there in the future), I just think Nitro is more sexy :) See all comments

Online Setting editing!

So, even this is done, and seem to work fine. Now, I'd like to provide the user the chance to save it, and I'm using an old patch wich gives me @Configuration.to_yaml@ and @Configuration.save@, except that it does not work :(. It seem that the tests were not complete, and even if they pass, the dumping phase fails due to some anonymous classes in the real configuration, which is not available in the restricted world of testcases. As for the UI: I settled on a new page (should it be a new controller?) wich shows a table with the settings and a text field near it, with a submit button. The user can then enter `contents`true" or "false" or a numeric or a string, and everything will work fine. This leaves the door open for other types, i.e. hashes, or arrays, but with the limitation of using only base types in them. The only other chance would be using @eval@ but.. well.. it is *so* evil.. So, for now I'm just showing numerical, strings and boolean values to the user, because those are easy to type/modify, and are those that make the most sense to change from the interface. But how to read back the values? This is a complex problem for me. Python has a great advantage over ruby here, because almost all the base types have a string constructor, so you can just do @someclass(somestring)@ and it will use the builtin parser behind the scenes. But ruby lacks this (and sadly, lacks a @BoolClass@), so for now I'm just adding this kind of constructor to the existing classes. Suggestions on better handling of this are really really welcome. Another thing: it would be nice if @Settings@ where @Comparable@ so I could sort them (and possibly make @Configuration.settings@ return the ordered list). Trans, if you ever read this please tell me what you think :) See all comments

Settings+scaffolding helper+ css patch

Ok, for who wants to try it out, "here is the new patch":/settings.patch.gz including all the stuff I've been working up to now, or at least the part of it I've record'ed :) I've put up a small stylesheet and imported some icons.. well, they're not that nice as of now, so help is very welcome. Btw, before we include those icons (tango package) we would need to add a line to the LICENSE, because they are attribution-share-alike. The to_yamlization is not active because of the problems with it. Making scaffolded stuff inherit from AdminController is not active either because of my inability to have it working. Damn. See all comments

Ruby CookBook Review

Fellow Rubyist (and my favourite Smalltalker :) Giovanni Corriga just did a "review of the recently published Ruby CookBook":http://ruby-it.org/pages/Ruby+Cookbook#1165 from O'Reilly. Seems something to keep on your desk, even if I need to check if I can find it on Safari, because my desk has no free space for more stuff ;) I have to admit, though, that I would have preferred if the CookBook had eveolved like "the python one":http://aspn.activestate.com/ASPN/Python/Cookbook/ , where everyone writes some recipes and put them online, and then some editors do the choosing, typesetting and so on. More community oriented and a resource that remains available for everyone. See all comments