KNotations

We still have much work to do in documenting the features and tricks of managing and customising knotes for site-admin folks. There's also a lot of work remaining to be done in making end-user capable skinning interfaces for knotes blogs. But let's not forget that, being based on Zope, Plone, ZPT, python etc, knotes is already pretty easy to adapt and customise for those with some ZMI skills.

By way of illustration: yesterday morning one of our regular end-user advocates got in touch to complain that the external links in this NGRF weblog entry were not opening in a new window. Sigh. We'd more-or-less forgotten that this was an outstanding feature-request - being web-standards types, we personally hate the idea of targetting external links into new windows and feel it is the user's choice. But we know that it is an important feature for a lot of site managers; in fact we've been through several iterations improving the spoecial handling of external links in the main site content of the NGRF.

We had a good javascript solution in hand, but needed to get that javascript code into the NGRF blogs, and have an onload handler added in those blogs, without stepping on the default behaviour of knotes blogs in other sites. So we made the following simple changes:

• In the product, we added a dtml-var include near the end of the blog_utilities.js.file, to test for the existence of a site-custom javascript file and include it if it exists (the name of that file is knotes_blog_utilities_extras.js).
• We added a custom version of that file in the NGRF portal skins / custom folder, and added the javascript code and onload-handler addition there

Since a little bit of CSS was also required by our external-links-handling solution, we added that rule to a site-wide custom css.dtml file which is included if present in the main screen css for knotes weblogs. Note that the knotes screen css includes both a site-wide custom and a blog-specific custom css file if present:

• knotes_weblog_weblogCustom.css.dtml
• knotes_weblog_sitewideCustom.css.dtml

All in all it went very quickly, with the change to the product being the time-consuming step. We'll add a similar customisation hook to blog_forum.js.dtml, and will have a close look at the customisation hooks architecture when time allows. We welcome feedback from site managers and developers about how best to add hooks to make it easy for them to meet the needs of their users.

By the way, you may want to check out the little javascript solution - it is a big improvement on the default plone link-scanner because it offers the user extra information and an option to over-ride the behaviour ( a rollover that shows the destination content type and a checkbox for 'open in new window' ). So users can opt out of the new-window behaviour for particular content types (for instance, pdf files may download to their disk in their browser's settings), and the visitor at least can see that the link will open in a new window if they do not untick the checkbox. We'll happily share the javascript code snippet if anyone wants it (about 7K of js).

We've just deployed a new method of handling comments by non site-members. If a comment is submitted 'anonymously', the email address given in the comment form is sent a confirmation email. If the special link in that email is not opened, the comment is not posted (see the screenshot). When submitting an 'anonymous' comment, a user can optionally ask to be registered as a member while the server is at it (by entering a username), in which case the confirmation email includes a server-generated initial password. Both anonymous commenting and joining-while-commenting are under control of site and blog properties. We're aware that some sites may have exotic registration policies or restrictions: in those cases the site manager should turn off the ability to join while commenting at the site level, using the KNotes tool in the ZMI.

I'm now beginning to experiment with having a one-form-fits-all add-comment form rendered directly into the one-entry views (most blogging systems do something like this). That form would cover all the cases: [I'm a logged-in-member; I'm a not-logged-in member; I'm not a member but want to join while commenting; I'm not a member and do not want to join].

We'll still include links for creating 'full-featured' discussion items (with extended text, categories etc), especially in the soon-to-come forums views. Watch this space ;o)

Sigh... I really didn't want to have to spend the time to implement this, but we decided it was made necessary by the nastiness of some of the trackback spam we've been seeing.

There is now a property in all KNotes weblogs, as well as a sitewide setting in the tool itself, for controlling whether trackbacks need approval before they are displayed. By default, this is set so that only approved trackbacks display. We will shortly be providing a weblog properties interface for managing this and other properties at the weblog level. The sitewide property is set through the tool in the ZMI - kndiscuss_sql (KNotes Config Tool). The sitewide property is acquired if the weblog property is not set.

There are new interfaces for moderating and managing trackbacks. See the screenshot for an example of the manage-trackbacks TTW (through the web) interface. If there are pending trackbacks, managers will see a portlet (in Plone content) or a sidebar (in KNotes content) for Managing Trackbacks. The number of pending trackbacks is listed there, along with a link to the TTW management interface. The manage-trackback RSS feed noe includes an indication of the workflow status of each item. We'll be adding links from there directly into setting the status - currently there is a link to delete each trackback but not for approving or rejecting. We'll also be adding interfaces for managing trackbacks wherever they are displayed ( in detail views, sidebars, portlets and plone-content comments views); and will also consider an interface for non-managers to report offensive trackbacks.

By the way - We've also been making a lot of progress on other aspects of KNotes recently, but that will have to wait for another entry here :O)

Phew! Trivial task turns into testing trial... I've just changed the layout of links in the Subscribe sidebar, so that the whizzy new RSS-2 filecasting, main-content feeds were exposed to the feed-consuming public.

I also took the opportunity to add a link to a new bit of stub content explaining all the formats and options ( there are loads of undocumented feed sources at the moment, for instance categories). I also flagged the default feed ( the RSS-2 that will be auto- discovered) on the assumption that it would be good to have one simple choice, and also to indicate for users with auto-discovery which of the feeds they will get.

Funny how these little bells and whistles can seem o=unimportant until you start creating content yourself :o)

I've long known that we should do something about logging the pings hat the trackback machinery attempts to send out. A trackback 'ping' is a little request we make to a trackback-aware server's "trackback:ping" url for their content we are linked to. That request includes the data about our own content. The other server can, if so configured, use that data - for instance to display a link back to our content's commentary on theirs.

I'm about to post a short overview of our plans for dealing with trackback spam, and realised that before going into those measures, I should first review the behaviour - current and planned - regarding authenication for RSS/atom in KNotes.

First - what is the issue? Basically, manegers can use the permissions form for a KNotes weblog to make the blog readable only to certain members. Likewise, a parent Plone folder could have been given a workflow state of 'private'. In either case, through-the-web pageloading requires authentication by a member with the correct permissions.

But what about the RSS/atom feeds for that private, members-only weblog? If a snoopy and savvy person wanted to type in the url for an RSS feed for the private weblog, surely they should not be able to read content in their news-reader which they could not read in their browser?

No, they should not. And at least some news-reading clients respect that. For instance, NetNewsWire offers username/password properties for a subscription, and will present an interface for entering them if the subscription demands it with its http response header.

And KNotes' RSS and atom feeds will not return content unless given an appropriate username/password when the request is made on a zope object which would require authentication for viewing through the web. Try it... if you subscribe in netnewswire to a private weblog, you'll have to enter a username an password ( in the get-info dialog for the subscription ) in order to fetch content.

BUT there is still work to be done:

Demand authentication nicely

We need to have the private feeds send back an authentication demand header rather than the error they currently do. This is a very small job but needs to be scheduled,

Return '' for nested private content in public feeds

More important by far: If you subscribe to a KNotes feed with '?include_discussion=1' -- ie you want to get nested content in the feed -- you can read nested private content. At the moment, privacy is only 'ert' at the level of the object the feed is called on. The content of the feed is assembled with an SQL query, so zope-wise permissions are not taken into consideration when grabbing the item content (and we definitely want the SQL speed). What we need to do is to impose a very simple policy: content which is not public through the web should not appear as nested content in any feed.

This policy would be draconian but safe. The SQL database kndiscussion table rows can 'know' whether or not "some" authentication is required (but cannot of course encapsulate zope's complex acquirable permissions, so cannot know whether the current request should authenticate against a row). But, since a row can know that its content is not public, it can have its feed content an empty string except when called directly on its parent object... in other words, a different query would have to be called for the non-nesting case

As you can see, some changes to the SQL data model are required in order to prevent non-public content from appearing in feeds other than those called directly on its parent. That means work, and will have to wait.

In the meantime, beware that private content could be sniffed by savvy snoopers. Personally, I would resist privacy anyway, but I appreciate that it is very important to some of our own users - and we will attempt to effect correct behaviour soon after 'release' :o)

Yesterday I posted about planning a 'release version' of knotes. I'm going to try to post at least once every day on this subject - heaven knows there is enough to write about! We use the FirstClass groupware system for our intranet. I send hundreds of messages a week to firstclass 'conferences' we've set up for documentation. I'm going to try to post some of those here instead.

As an aside, though, let me re-state the bleeding obvious: if you want to install KNotes in its current version, please do. We employ KNotes in as lot of sites now, and could not imagine living without it. But if you can wait a while, you might find life a bit easier. In either case, if you are thinking of trying KNotes at some point, please let us know.

We have to make some decisions about what exactly makes an issue a pre-release one. We already know of a lot of issues, minor bugs and feature requests, and we already have extensive plans for improving and extending KNotes. Most of those issues, bugs, features and improvements should be made post-release, but some of them would be much better completed before too many other sites were deploying KNotes in anger. I think these can be roughly categorised as follows:

After many, many delays due to illness and distraction, we're going to start putting together a practical roadmap for KNotes releases. In this post I'm just annotating an earlier post I made in February to point out what got done and whether priorities have changed. Bizarrely I note that the date posted is just a few days before I fell ill, but that the first tasks in the list did get done. I must have been pretty effective in those days :o).

Here is a hastily thrown-together list of issues I feel ought to be resolved before we publicise knotes. I'll be adding to this list, and annotating it, over the next few days.

OK - the list I made then was:

• Cleaner css/image skinning for weblogs - DONE - but user requests and practice since then have forced us to add 'skinning interface' to the release-1 features
• Complete fast_folders' move of 'no-wrap' from custom main_template to custom skin_path - STILL ESSENTIAL, JUST NEEDS SOME CAREFUL APPLICATION OF AN EXISTING SOLUTION
• Complete the migration to fast_folders as view for KNDiscussion - AFTER VACILLATING ON THIS I NOW AGREE THAT IT IS ESSENTIAL
• Complete the discussion-view for weblogs - DITTO

So, the top item on February's list got done (in a day or so :O), but we've since had so many demands for a skinning interface (other than the ZMI + ZPT+CSS knowledge) that I feel I must add some basic support for end-user skinning before we call KNotes 'released'. I'll write some notes on what 'basic' skinning support means in another post - choosing a stylesheet from options, setting (ugh!) background colours and banner images TTP, etc

The other items remain important, and remain undone. The fact that they remain undone is not surprising. I've only been back on my feet at all for a month and a half, and that has been totally occupied with catching up with project work, essential writing and putting out admin fires. Oh - and getting up to speed on e-portfolios and social bookmarking / social software integration issues. Weirdly, in the three months I was ill, the state of internet applications for the masses seems to have ratcheted forward a good big step after years of what looked like frustrating inertia to me :o)

One final point on those pre-release to-dos from February. At the time, I had planned to create a fallback, plain-html, accessible discussion view for the blogs and discussions, but to throw most effort into a fast javascript-savvy client loading data only and allowing many contextual actions without overall pageloads. In the months since, I had started to worry that the work to complete the javascript-savvy interfaces was - though 98% done - still a post-release issue. Since then, though, our users have in effect demanded faster and more convenient interfaces. Also, the 'AJAX' webapp style has begun to get some hype and support, which has made me less apprehensive about the accessibility issue. We will have a totally javascript-immune, accessible to all devices interface for browsing discussions and replying to them. But for early releases I am willing to keep that extremely crude and to put all the effort into a really fast and transparent application-like discussion interface.

More to come. Steve is back from Canada this week; we'll try to make a do-able and understandable roadmap within a few days.