Oh my, Barnaby what have you become?

This election has not been kind to you Barnaby, in fact we have seen you metamorphose from straight down the line reasonable peoples politician reflecting your grass roots to a bitter and desperate party politician and one that has gone from attacking policy to one attacking persona’s.

Yesterday I heard you speak after Julia announced digital health reforms in Townsville to work in with the National Broadband Network to deliver doctors consultations to regional Australia where it would not be otherwise viable and instead of this being music to your National ears it invoked the scorn and dismal displays I have come to expect from you recently.

Lets look at this closer shall we, yesterday the Senator said;

“The Labor broadband plan could have been authored by a poor pretender to Hans Christian Anderson. Australia would have to believe in fairy tales to believe that the Labor Party can deliver this…”

Hmm I seem to remember a different tune you sang not to long ago.  In April of 2009 just after the NBN announcement by Kevin Rudd you told us;

“How could we disagree with something that is quite evidently our idea,”

“This delivers a strategic infrastructure outcome.”

“It is vitally important that the National Broadband Network gets to the corners of our country where the market has failed, at a price that is both affordable and a service that is comparable.”

Now hang on why did you say it was your idea?  Oh thats right the good Senator once worked with NSW Nationals Senator Fiona Nash on a white paper on telecommunications policy, in which a major tenant was the creation of a national fibre optic network, and restructure of the industry in recognition of the crippling lack of infrastructure in the bush.

The Senator unveiled this white paper to Parliament and of course the Howard government promptly ignored it, in the vain hope that market regulation would do the job for them but of course it won’t the numbers just aren’t there to justify the cost of delivering needed services to the bush.

See this is the problem with governments that think they need to be run as a business, and that everything has to balance on the ledger.  Not every thing is economically viable, and that’s okay.. it wasn’t viable for a private company to provide phone services to all corners of Austalia, nor was it to have bank branches everywhere, nor would it be to have private hospitals.  Where the market cannot sustain or make services pay thats when we need the government to step in to pick up the tab.  The services available to tax payers in metropolitan need to be available equally to tax payers in regional community centers.

I’ve also got the shits with Barnaby and co sprouting the reasons for having a NBN is so some kid can watch youtube faster, that is a total crock of shit.  I couldn’t care less about the mum and dad users of the Internet having fibre, but when I have to pay over 100k a year for a substandard 10mbps business grade connection just because I am not inside the CBD there is something amiss in this country.  That is someones job right there, and it is replicated all over the country.

The need for faster wider bandwidth for business is growing day by day and in 5 years the lack of infrastructure will cripple us because it already is.

Refugee’s and the Myth

I read in the media and more concerning hear from friends about  the great swarms of illegal refugee’s coming to our shores.

I’ll be the first to admit we can and do get taken advantage of by a portion of these people but that doesn’t mean you stop doing the right thing and I would like to point out that we don’t have any such thing as illegal refugees in Australia.

Under the United Nations charter for refugees of which Australia signed both the protocol (1951) and convention (1967) under Menzies then Holt, both then Liberal governments.  We have an obligation under international law to accept and protect them.

There is a cost to that but when is there not a cost associated with living up to expectations?  Just by the by the Howard Government actaully holds the record for the most boat arrivals. I can’t say I agree with how we handle the boats but we have an obligation and there is no easy solution to it.

The nation of Nauru is not the answer, the reason Howard choose Nauru is to avoid our obligations under the UNHCR.. and well to me sherking your responibilties is just not how we do things nor is it an example we should want to show our children. We need to do more to take care of our own for sure but one really doesn’t have anything to do with the other.

It’s a myth that Refugee’s are treated better then our other welfare recipents, Centrelink and Medicare benefits only become available to refugees once they are issued a protection visa and gain permanent residency.  ie they are legitimate refugees, and as legitimate refugess is there really any Austalian out there that would turn their back on someone in need?

They then get the exact same benefit rates as any one else, not more.  They certainly do not get free houses or cars…

I’d like people to try and understand that the issue of refugee’s is not an issue of border protection, nor is it something we can stamp out with policy.  We have more people over stay their visa’s then we do arriving by boat but I hear no mention of that in the media. 

To want to risk your life and that of your children to these people smuggilers tells me you  are desperate, and we should remember often in the parts of the world these people are coming from there is no legal means to reach us.. it’s not like they can just stroll down to the local embessy.

Historically the flow of boats heading for Australia coincide with civil unrest in the region, this is something fairly out of our control and can’t be prevented by policy.  All our policy can do is control the funnel, under Howard it was funneling the boats to detention centres outside the immigration zone beyond the eyes of the UN.

Do I have the answer no I do not, and I doubt a truely universal one will ever be forth coming.  What I do know is that we as Australians have stood up and said yes we recognise that refugees are a class of people that need protection and we will do that.  We are a compasionate people and the ideal of helping out those who can not help themsevles is ingrained it us..

This makes me proud to be Australian, and I would not welcome any policy that would dimmish that, even if it means being scammed by 9 in 10 people.  If just one is legit we are doing the right thing.

http://www.refugeecouncil.org.au/current/myths.html

Twitter

Okay since someone just asked me if I have twitter, the answer is no I don’t.. well rather I didn’t but I do now, so if you want to get in contact with me or comment on something you can now tweet (is that the term?) me @idiomatically

Dear Ms Gillard

So it would appear all our hopes and desires have been dashed and we are still lumped with Senator Conroy and mandatory internet filtering.   It disappoints me somewhat that Ms Gillard can’t see the forest for the trees.  I’ve said it before there are 16 million internet users in this country and I would hazard a guess that 80% of them oppose mandatory filtering, that is a very very large voter pool to be ignoring and not taking seriously.

This fear, uncertainty and doubt card Senator Conroy and now the Prime Minister are playing is just that it’s smoke and mirrors, it’s not real.. there is no great threat to our children coming out of the internet, no more so when Robert Menzies tried to have the communist party outlawed in the 1960’s.. all there is fear, and as we should all know fear is the mind killer.

I read a quote this morning from Prime Minister Gillard that went like this;

“I’m happy with the policy aim and the policy aim is if there are images of child abuse, child pornography … they are not legal in our cinemas, you would not be able to go to the movies and watch that … you shouldn’t … no one should want to see that.”

When you look at a policy aim and outcome, you need to assign a metric to measure this and decide if it can successfully achieve it’s stated aim.  Putting aside all moral debate about free speech and democracy , I want to look at this policy in it’s simplest form can it achieve the stated outcome?  The answer is a resounding no, Senator Conroy has already stated on more then one occasion that any technically inclined person can circumvent the filter, that numerous transmission mediums found on the internet cannot and will not be filtered.

The filter will only listen on http ports, this is just one of a dozen protocols used on the internet.. there is no filtering of p2p, ftp, embedded media sites, external proxy services, ssh, gropher, or newsgroups.  Nor can it filter encyrpted SSL https traffic.. this is like designing a car that can only turn one wheel out of four and expecting it to drive in a straight line.   Forgetting all that still, the key metric we need to use if this is really about the children is simple how many more arrests will be made and how many more children will this filter prevent from being abused?  I will go out on a limb here and say not a single one.

Thing is the scum bags that perpetrate these crimes operate in the shadows, their insidious nature means they hide and lurk they do not register tld’s / domains  and put up big ole billboards on the internet that say “find us here”.  I challage the Senator and the Prime Minister to sit down with the Australian Federal Police crack open Google or Bing and start searching for this child porn content I promise you a google search will not find it, it’s not made to be found by average joe punter.. there is no accidental exposure to it.

Coming back to the Prime Ministers quote, she is right it’s not legal in our cinema’s, you can’t go down to the dendy and watch it and rightly so, but does she really honestly believe that this content is not already here or that cinema’s are the method of choice these vial creatures use to watch or distribute?  Where is the great mandatory filter on the movie industry, why are customs officers not inspecting every single dvd, blu-ray, vhs that enters the country for it?  Because it’s a waste of time and they know it.

When you let fear govern your policy you are out of control.. do we have so very little faith in our police investigators and our criminal justice system that we need to start taking stabs in the dark?  We have a method of dealing with this filth it’s called the cyber crimes unit.  We need to take the resources being wasted on this bad policy and apply it to where we can quantify the outcome,  give the police more resources to track, apprehend and prosecute these people.

A mandatory filter gives a false impression of safety to parents and children a like, fact is most predators on the internet operate out of chat rooms and instant messaging clients again something the filter can have no effect on.  For me the most scary thing happening on the internet at present are the social networking sites,  they give away far too much personal information to perfect strangers this makes it easier for predators to pick their targets, not to mention it’s been the greatest tool for identity theif in the last decade.  To give you an example a friend of a friends daughter who is 11 I might add had her full name, address and mobile phone number listed for all and sundry to see.

We need a policy of greater education on how to be safe on the net, an opt in filter so parents can decides what their children can and can’t access (plenty of which is legal content), and a robust criminal investigation unit with the resources to track these predators from one side of the globe to the other.

Unfortunately we have instead a half wit for our communications Minister, and a Government unable to understand the real nature of the issue that is either too pig headed to work out it’s bad policy and back away from it or think of it as a non-issue to voters. The sign of a bad Government is one that cannot admit if got it wrong and then set about correcting it’s mistakes.  I feel I have been backed into a corner I can not support the Liberal party as they are only liberal by name and not by nature, nor though can I support a Labor party that peruses the mandatory censorship of it’s people.

The filter in itself is insidious in that it blocks content that is quite legal in Australia based on what some person deems is amoral rather then illegal.  We are smart enough to self censor, I know what I do and do not want to see, as are most people but by god if I am denied the right to view and research legal topics why don’t we just light the book pyers of the 1940’s because that is what we’ll have become.

Modify your PATH on Linux

Normally this only comes up these days if you are compiling your binaries or have downloaded some out side your package management system.   I come from Solaris so I like to use /opt for all my self compiles and I prefix that with my initials ie /opt/mwd/bin this way I know it’s software I’ve compiled 6 months down the track when I am cursing it’s not got support for something I now need.

I use a multitude of linux distributions based on the hardware or purpose of the machine in question, ie servers are normally centos for greater vendor support or debian, and opensuse for my desktops.. I find this method works more or less for all the different linux flavours kicking around.  I am assuming like most people bash is the shell of choice.

As most will know the PATH environment is a variable that get set every time you spawn a new session, login or source it by a group of system configuration files.   These configuration files are basically read in the following sequence;

  • /etc/profile,
  • then every /etc/profile.d/*.sh file that your user has access to,
  • then /etc/profile.local if it exists (by default it won’t),
  • and finally by $HOME/.bashrc and $HOME/.profile

Every time you spawn a session or shell, the new shell process will inherit all the environment variables that have been provided by the configuration files above, what I mean is they have a flow on effect in that no one files dismisses the previous they build on each other.

When it comes to setting your personalised PATH environment you really don’t want to use a configuration location that will get clobbered in system updates so /etc/profile is out, don’t touch it.  I also don’t like using $HOME configuration files as I have users on my systems that I generally want to make this software available to and since /etc/profile.local doesn’t exist by default and will be globally read by all users on the system I choose it 🙂

If it was just for your user, then apply the change to $HOME/.profile other wise do as I do and apply the changes to /etc/profile.local.

I find the quickest way to achieve this is to simple issue the following command;

echo ‘PATH=/opt/mwd/bin:$PATH’ >> /etc/profile.local

(remember mwd are my initial this could easily be foo)

I would like to mention that in my example here we are appending to /etc/profile.local by using “>>” instead of just “>” as this creates / overwrites existing files.  This is a safety feature, as on some distributions this file may already exist and we might not want to blindly clobber it.  Also as stated previously the process inherits any variable that has been set previously by an earlier configuration file so in this case we do not need to “export” our new path.

Once you have made this chance you most likely won’t feel like having to respawn your session or logout and back in, so we’ll just source our new configuration with;

source /etc/profile.local

It’s that simple, one last thing.. if your linux system only has one user and that user is you and you are compiling your own stuff it may be easier just to install to $HOME/bin as most modern distro’s will read this path already in the default profiles.

Baked Jam Roly-Poly

Ingredients:

  • ½ cup cold butter
  • 1 cup self-raising flour
  • Small amount of cold water
  • Jam at room temperature (flavour of choice, although it is traditionally made with plum jam, use SPC Dark Plum for best result!)
  • ¼ cup sugar
  • 1 tablespoon butter
  • 1 large cup of boiling water
  • Extra flour for flouring bench and rolling pin
  • Extra butter for greasing dish

Method:

If mixing by hand:

1.      Place the flour in a medium-sized bowl.

2.      Cut the cold butter into small chunks and add to flour.

3.      Rub the butter through the flour until it resembles coarse breadcrumbs.

4.      Add cold water a bit at a time until it forms a ball of dough.  Be careful adding the water so that it doesn’t get too wet.  If it is too wet knead in a bit more flour.

If using a food processor:

1.      Place the flour in the processor bowl.

2.      Cut the cold butter into small chunks and add to flour.

3.      Put lid on processor and process until the mixture resembles coarse breadcrumbs.

4.      With the processor running add cold water a bit at a time until it forms a ball of dough.  Stop machine as soon as it forms a ball.  Be careful adding the water so that it doesn’t get too wet.  If it is too wet add in a bit more flour.

Either method:

5.      Sprinkle flour over bench.

6.      Tip dough ball onto floured bench and knead for a few moments.

7.      Roll dough out using a lightly floured rolling pin.  It is best to roll in one direction only rather than backwards and forwards.  The finished dough should be about ½ cm thick and roughly rectangular in shape.

8.      Spread with jam taking care not to break the dough.  It is best to use the back of a spoon to spread the jam.

9.      Starting at one end of the narrow side of the rectangle begin to roll the dough into a log. Turn in the ends.

10.  Use extra butter to grease the inside of a glass or ceramic baking dish with deep sides.  The dish can be rectangular or oval as long as it is long enough to hold the roly-poly and it is deep enough to hold the sauce.

11.  Place the roly-poly into the greased dish.

12.  Place the ¼ cup sugar into a small bowl and add the tablespoon of butter.

13.  Pour the cup of boiling water over the sugar and butter and stir until the butter melts and the sugar dissolves.

14.  Pour sauce mix slowly over the roly-poly.

15.  Place in pre-heated moderate oven (180°C) for about ½ an hour or until golden brown on top.

16.  Serve warm (some muppets even use custard.)

That filter policy

I’d like to share with you all a letter I sent to my local member who then passed it on to Senator Conroy as it was beyond my members abilities apparently,  it has been passed on but as yet there has been no response after many months so instead I thought I would just put this out there.

Dear Graham,

Thank you for your response, however I am disappointed with some elements. Perhaps you have been mislead as some of your reply appears not to address the facts as far as I can see. I now know your party line but I don’t know how you my member of parliament feels about all this, so I will assume you will support the filter rather than oppose it.

How has the ALP managed to do a direct back flip on this issue when in 2003 the Labor Party opposed filtering at the ISP level?  Labor senator Kate Lundy stated;
“Unfortunately, such a short memory regarding the debate in 1999 about internet content has led the coalition to already offer support for greater censorship by actively considering proposals for unworkable, quick fixes that involve filtering the internet at the ISP level.”

Didn’t I see Kevin Rudd slamming the opposition just this week for such things? There are nine points of reference in your reply I would like to visit and I do apologise for the length of this response but I believe it is warranted to put forward the view I wish you to understand if not support.

1.  “The ability to use online tools effectively”
How can one use a tool to its fullest while it’s being interfered with? In the world of information technology we are always trying to knock down and remove bottlenecks from systems, and the policy of adding a mandatory filter is one giant bottleneck that in the real world will result in poor performance. Take our gateway bridge; why were the manual toll booths removed? To improve traffic flow and performance, and the web or information highway as it used to be called is no different.

Replace the word traffic with data and you have the same thing, and in this case instead of removing the barrier (i.e. toll booths) you are in fact adding one and while you’re at it you want to do a vehicle inspection at the same time.  How can this not slow it down? As Steve Dalby, chief regulatory officer at iiNet, who said filtering the internet at the ISP level, as opposed to installing filtering software on the home computer, was unworkable and would “affect the performance of the network quite significantly”.

2. “The pilot, and the experience of ISPs in many western democracies, shows that ISP level filtering of a defined list of URLs can be delivered with 100 per cent accuracy. It also demonstrated that it can be done with negligible impact on internet speed.”  Sorry but let’s put this into some context, I return to my earlier corollary of the Gateway Bridge. If I was to do a traffic flow test on the bridge using only 0.0625% of its daily traffic would the results say the traffic flowed perfectly and there was no need to remove the toll booths? I dare say they would, but we all know that would not be a true reflection of the case.

According to the International Telecommunication Union as of February 2009 there were 16,926,015 internet users in Australia (79.6% of the population) The figure of 0.0625% I quote above is the ratio of people who took part in the Enex Testlab live trial and consequently some people think using those numbers is folly, since they cannot effectively reflect the case. Statistics experts such as Dr Daniel Johnson at QUT and Professor Ron Hyndman at Monash University have correctly called out this report as being unscientific and extremely flawed.
http://www.arnnet.com.au/article/312845/statistics_experts_label_isp_filtering_trials_unscientific

Enex Testlab’s managing director, Matt Tett although admitting the testing did not comprehensively investigate what effect user load would have on the filters, claims this is not a factor saying;
“The number of participants isn’t necessarily relevant. The load is not relevant, absolutely not. The number of people who are on the filter itself, the number of people on the system and whether they’re being filtered or not, is irrelevant”

A pity the rest of the industry doesn’t share that view, I draw your attention to the warning from Australia’s peak technology group the Systems Administrators Guild of Australia (SAGE-AU) to the Federal Government that any internet filtering laws will fail to work because the technology was only tested on slow broadband services. Ms. Donna Ashelford of SAGE-AU said the report from the Internet filtering trials was unclear about sample sizes, which was vital to understand if its results were statistically significant. “Large numbers of participants would obviously have a greater impact on performance than a smaller number,” she observed.

Ms Ashelford further stated:
“The only widely quoted figure from a test participant was a handful of clients, which did not produce any significant load. Significant sample sizes are essential to understand the effect that Internet filtering may have on service performance. Another concern is that the report admits that ‘a technically competent user, could, if they wished, circumvent the filtering technology’. Anybody who uses Google could find ways to access censored content.” Ms. Ashelford said every filtering solution tested had failed under “heavy traffic” sites on the Internet including YouTube videos already blacklisted by ACMA (the Australian Communications & Media Authority). “The results show that none of the filters coped with widely used technologies such as peer to peer, chat rooms or instant
messaging,” she said. “No false-positive data was provided for ISPs which were only blocking ACMA-prohibited URLs, which was in the terms of reference.”

Indeed the sample group of users in the live pilot was less than 1% as compiled by the participating ISPs, and this is not a representative sample, particularly since some customers complained about over-blocking, and withdrew from the trial. One example was the blocking of the pornography website redtube.com which is legally accessible in Australia yet somehow made its way onto the AMCA blacklist, I thought this was only meant to be RC material?

While still talking about speed and performance I would like to inform you about an internet phenomenon known as the Slashdot effect; The Slashdot effect, also known as slashdotting, occurs when a popular technology news website links to a smaller site, causing a massive increase in traffic. This overloads the smaller site, causing it to slow down or even temporarily close. The name stems from the huge influx of web traffic that can results from a technology news site linking to other websites.

This is what happens when you try and push too many connections into a smaller pipe that can’t handle the load, Slashdotting is unintentional but there is another form known as a Denial of Service attack and in fact a number of Federal Government websites have fallen victim to these this month as part of ‘Operation Titstorm’ by the group anonymous. Again this is what happens when too many connections are made and the system can’t keep up.

Kind of like the grid lock on the Gateway bridge which the State is so eager to alleviate. By forcing amandatory ISP level filter on the industry you are in fact forcing grid lock into the system, a man made bottleneck for no gain what so ever. Under no load of course the results will say it has no impact, but you throw everyone at it and it will come to a smouldering wreck.  Even Telstra back this up by saying; “As a general rule, there appears to be a relationship between measures to counter deliberate circumvention and impact on internet performance — i.e. stronger circumvention prevention measures can result in greater degradation of internet performance,”  According to the report, Telstra found its filtering solution was not effective in the case of non-web based protocols such as instant messaging, peer-to-peer or chat rooms.

“Enex confirms that this is also the case for all filters presented in the pilot,” the report reads. “Telstra reported that heavy traffic sites could overload its trial filtering solution if included in the filtering blacklist. This is also the case for all filters presented in the pilot.”  Can you see that heavy traffic sites could overload the filtering solution? Think about that for a second, going from ten thousand users in the trial to sixteen million real world users.  Do you think that might be considered heavy traffic?

3.  “In particular our approach has been informed by the constructive input of Australia’s four largest ISPs…”

Why is it none of these were made part of the initial testing and there has been no consultation with them or peak body groups, rather the Federal Government is pushing this on the industry without listening to it.  Case in point Nicholas Power of Highway 1 when asked his stance on ISP level content filtering stated:  “We were one of the unexpected participants of the trial and it was an interesting process. Our intention for participating was to find out more but in reality, this did not eventuate. We found it to be as much of a black box as those people on the outside, who were asking us questions.

The idea that ISPs should police their users is something we are against. As a smaller ISP, we’ve seen the significant cost required just to implement a trial. In the end, I don’t think this is a technical race that can be won.  Optus:  “Optus would rather work with Government to legitimately understand the implications of this type of filtering process, than have it mandated”
Out of the top fourteen ISP’s in Australia only Telstra and iPrimus support the introduction of the filter, and all of them say it’s a waste of time. But even then iPrimus the largest ISP to take part in the trial by Communications Minister Stephen Conroy only support it under duress.

General manager of marketing and products Andrew Sims said while web filtering services suited some families, they should not be made compulsory.   We’ve got instances of that around the world, particularly in China where the government forces filtering upon their population, my professional opinion is I don’t really believe that forced filtering is a good option.”  Then there is iiNet one of the top three ISPs in the country.  in a statement:  As iiNet managing director, Michael Malone, said  “We are not able to reconcile participation in the trial with our corporate social responsibility, our customer
service objectives and our public position on censorship” “It became increasingly clear that the trial was not simply about restricting child pornography or other such illegal material, but a much wider range of issues including what the Government simply describes as ‘unwanted material’ without an explanation of what that includes.”

I have also attached for you a letter to the staff of PIPE Networks from their CEO Bevan Slattery about why they are black listing the trial and filter, it is interesting reading. This policy of mandatory internet filtering does not have the support of the industry nor does it have the popular support of the Australian public.

4. “Introduction of mandatory ISP level filtering of Refused Classification (RC) content in order to reduce therisk of inadvertent exposure”
If that is the case could you please explain to me why when Greens senator Scott Ludlam asked questions related to the trial at the end of 2008, one of the answers the Federal Government provided in January 2009 was that 674 out of 1370 blocked sites on the mandatory list relate to child pornography; 506 sites would be classified as R18+ or X18+, despite the fact that such content is legal to view in Australia. That means that 37% of material on the AMCA blacklist is NOT RC content.

5.  “…can be used to distribute material which is not acceptable to most Australians, particularly children.”
This to me seems more like Senator Conroy’s “unwanted material” then RC content and we have already seen that legal content sites are being blocked on the AMCA list. I wish to point out to you the ranking of some sites in Australia according to Alexa the company for metrics and site rankings worldwide on the internet that are at odds with that statement. All of these sites contain unwanted material according to the Minister’s definition and some even contain RC material but hopefully you will soon realise how absurd it would be to block them;

Ranking 8 wikipedia.org – online encyclopaedia, RC material contained as has articles on drug use and euthanasia.
Ranking 25 imdb.com – film and television site, RC material contained as has reviews and images of RC films.
Ranking 51 pornhub.com – pornography site already blocked by AMCA blacklist even though it is legal.
Ranking 52 rapidshare.com – file transfer service, RC material, has patch and game files for RC games.
Ranking 53 thepiratebay.org – p2p site, Senator Conroy is on the record as wanting to block p2p sites.
Ranking 54 ezinearticles.com – online magazine, contains RC articles about drug use.
Ranking 57 youporn.com – pornography site already blocked by AMCA blacklist even though it is legal.
Ranking 58 redtube.com – pornography site already blocked by AMCA blacklist even though it is legal.
Ranking 62 livejournal.com – online journal, contains RC articles about circumventing copy protection etc.
Ranking 68 isohunt.com – p2p site, Senator Conroy is on the record as wanting to block p2p sites.
Ranking 96 torrentz.com – p2p site, Senator Conroy is on the record as wanting to block p2p sites.

With sixteen million internet users in Australia all of these sites are in the top 100 for the nation, which suggests they are very popular with many people. Do you really think they will support having their access to these sites blocked?

6. “Filtering technologies have been adopted on a voluntary basis by ISPs in a number of countries including the United Kingdom…”
Let’s talk about this one for a moment, the UK uses the IWF blacklist which will form part of the mandatory blacklist used in Australia. In December 2008, hybrid filtering technology implemented by UK providers caused disruption of Wikipedia operations in the UK when a Wikipedia page was added to the IWF watch list. When Wikipedia blocked UK vandals by their IP address, this block affected all users coming from these IP addresses.

As these IP addresses belonged to the filter proxies, some Wikipedia users in the UK, depending on their ISP, attempting to edit an article without a login name were blocked. Some proxies also collapsed under load generated by Wikipedia traffic.  After widespread coverage, the IWF removed the Wikipedia page from its blacklist, citing the availability of the image on other websites as a factor:
“IWF’s overriding objective is to minimise the availability of indecent images of children on the internet, however, on this occasion our efforts have had the opposite effect … We regret the unintended consequences for Wikipedia and its users.”  That doesn’t seem all that successful to me, how’s that working for you?

7.  “…URLs of known child abuse material…”
Isn’t this then an issue for law enforcement? If these sites are known, surely any judge can order them taken down and I think you will find the industry would be more than happy to black hole the routes they use. This is like saying we know there are criminals dealing drugs in that suburb’s park but rather then shut them down we’re just going to stop anyone on our beat from using that park instead.
That is nether law enforcement nor prevention.

8. “Enex Testlab, an independent testing laboratory…live pilot…ISPs of varying sizes and their customers”
How can Enex Testlab claim to be independent when they work for no less than thirty Government departments? A negative finding would be akin to biting the hand that feeds you and their lack of
completeness and flawed testing model guaranteed that couldn’t happen. The fact that Telstra’s trial outcome differed quite dramatically makes that plain and Telstra would support a filter if legislated. I would also point out again that only a minute number of ISPs customers took part in the so called live pilot.

9. “…transparency of processes that lead to RC material being placed on the RC content list.”
How can there be any transparency when Senator Conroy has vehemently defended his decision to keep the blacklist contents secret. Even the ISPs in the trial have said they are none the wiser about the process.  How can we trust a Government that banters about terms like “unwanted material” but doesn’t define what that actually means? We know it’s not just RC material, as many legal sites are blocked. Even political parties such as the Australian Sex Party have had their website added to the AMCA list. Why? Because the word sex makes an appearance?

I do not need my Government to tell me what I can and cannot look at, I am quite able to self censor material I do not wish to see and I am more than able to censor material I feel my young children should not be exposed to. There is really no such thing as inadvertent exposure, it’s a myth. No one accidently ends up at a porn or RC website; they have to knowingly follow a link, etc. In over 15 years of being on the net I can honestly say I have also never seen or come across child pornography, they doesn’t exactly advertise this criminal behavior and I day say they don’t even use the http protocol which is the only protocol the filters block.

Finally I would like to end with the UN Charter of which Australia is a founding member having signed it on 1st of November 1945.
Article 19, a Fundamental Declaration of the UDHR, concerns the Individual, and states:  Every Individual has the right to freedom of opinion and expression; this right includes freedom to hold
opinions without interference and to seek, receive and impart information and ideas through any media and regardless of frontiers.

The internet is just a new frontier. Has Australia forgotten its UN Charter responsibilities?

WinXP install from USB Key

So the inevitable happened today we had to reinstall a netbook for one of our staff, there are probably offical methods for doing this from HP and co they involve restore partitions and they like but well since I don’t like reading we have our own method for installing Windows XP sp3 from a USB Key.

First of all you are going to need a few things;

Working USB Key (must be about 1GB is size we use Corsair GT’s)
Windows XP sp3 media CD
Good copy utility like Roadkil’s

Hiren’s Boot CD (http://www.hirensbootcd.net/download.html)

I’m going to assume if you are reading this you are somewhat savy and able to burn a cd.

Step 1.
We need to make our USB key bootable, to do this we are going to format it and install grub4dos.

Grab http://www.hiren.info/download/freeware/usb_format.zip

Then Grab http://www.hiren.info/download/dos-files/grub4dos.zip

After these two steps you should now have a bootable USB key.

Step 2.
We now need to copy the Hiren’s Boot CD and Windows CD to the USB key using Roadkil’s (or some other method of choice).
First copy the entire Hiren’s Boot CD contents to your USB key, then create a new folder in the keys root directory called ‘WinXP’. Copy the entire contents of the Windows Media CD here.

You should end up with two directories on the key, a HBCD directory (about 180mb) and a WinXP directory (about 580mb).

Step 3.
Configure grub4dos by coping the files grldr and menu.lst from HBCD directory to the usb keys root directory.

Step 4.
Now that the USB key is setup we need to boot it. To boot from USB you will have to either set it as the primary boot option in the bios of the netbook or just hit the select boot function key as the netbook starts (HP is F9, most seem to be F12 though).

Step 5.
If you are not a muppet you will be looking at a dos menu screen;

Boot from hard disk, or
Launch Hiren’s Boot CD, or
Launch Mini Windows XP.

You want the ‘Launch Mini Windows XP’ option.

Step 6.
The cut down mini windows xp will boot pretty quickly and you’ll be presented with a familiar XP desktop enviroment. We now want to invoke the Windows XP installation but opening a command prompt. Navigate to the USB key drive and select the ‘WinXP\i386’ directory, right click it and select the “Command Prompt Here” option.

Once you are looking at the command prompt simply type;

“winnt32.exe /makelocalsource /syspart:C: /tempdrive:C: /s:.” without the quotes of course and hit enter.

This will start the first phase of the install by coping temp files to the netbooks local hard drive, just answer it’s questions till it completes then it’s time to reboot and start the install in earnest from the hard drive.

Step 7.
Reboot, boot from the hard drive as per normal and continue to install windows as you would any other machine.. ie click next a dozen times and drink a beer or two.

Enjoy.

Open Source is Inferior

Well at least it is according to the dumb bitch sitting next to me at the moment. Take about a red rag at a bull, what a stupid stupid statement to make. With that end in mind here’s a list of great open source / freeware applications for windows I love and they are certainly not inferior (in no particular order);

1. ISO Magic http://www.magiciso.com/tutorials/miso-magicdisc-overview.htm
2. 7-Zip http://www.7-zip.org/
3. Audacity http://audacity.sourceforge.net/download/windows
4. Avidemux http://avidemux.sourceforge.net/
5. BRL-CAD http://brlcad.org/
6. Bid-O-Matic http://sourceforge.net/projects/bom/
7. ClamWin http://www.clamwin.com/
8. Cream http://cream.sourceforge.net/home.html
9. Filezilla http://filezilla-project.org/
10. Pidgin http://pidgin.im/
11. Juice http://juicereceiver.sourceforge.net/
12. Mozilla http://www.mozilla.org/
13. PDF Creator http://www.pdfforge.org/pdfcreator
14. Gimp http://gimp-win.sourceforge.net/
15. Open Office http://www.openoffice.org/
16. True Crypt http://www.truecrypt.org/
17. VLC http://www.videolan.org/vlc/index.html
18. Utorrent http://www.utorrent.com/
19. WinSCP http://winscp.net/eng/index.php
20. Miro http://www.getmiro.com/
21. Putty http://www.chiark.greenend.org.uk/~sgtatham/putty/
22. Handbrake http://handbrake.fr/downloads.php
23. Inkscape http://inkscape.org/download/
24. XChat http://www.silverex.org/download/
25. Notepad++ http://notepad-plus.sourceforge.net/uk/site.htm
26. GnuUtils http://unxutils.sourceforge.net/
27. Scribus http://www.scribus.net/
28. Blender http://www.blender.org/
29. JDiskReport http://www.jgoodies.com/freeware/jdiskreport/
30. Speak Tools http://www.speaktools.com/
31. Eclipse http://www.eclipse.org/
32. InfraRecorder http://infrarecorder.org/
33. ImgBurn http://www.imgburn.com/
34. Clonezilla http://clonezilla.org/
35. KonBoot http://www.piotrbania.com/all/kon-boot/

Expanding Zimbra Ext3 Mail Store.

Recently I noticed that our primary mail store was becoming alarmingly large and running out of disk space. Origianlly 200gb seemed ample but fast forward a couple of years and well it’s vastly inadequate with 92% of the available disk space consummed and growing rapidly. Infact if it wasn’t for the GFC I would dare say we’d have well exceeded it by now.

Since the mail store is an iScsi volume mounted as a phyisical disk (dev/sdc) increasing the volume size was easy, I just allocated more resources to it at the SAN but that doesn’t really do much for us as the Operating System doesn’t know to use the additional space as we can see from;

Disk /dev/sdc: 429.4 GB, 429496729600 bytes
255 heads, 63 sectors/track, 52216 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdc1 1 26108 209712478+ 83 Linux

We need to grow our /dev/sdc1 partition to use these new disk cylinders. But before you start messing with any partition or file system the golden rule must be back up and not just once or in the same format when it comes to mission critical data, a mistake here is likely to get you in the shit at the least and maybe fired or worse.

So the next step is all this is to do a couple of “FULL” backups, the first I did was from within Zimbra itself this will not only compress the mail store but also grab all the LDAP info etc as well. This back up would let you deploy to another server instance if things went really bad;

zmbackup -f -s $server.domain.net.au -a all -t /mnt/SANBackup/zimbra.backup/ -z

where server.domain.net.au is the fully qualified domain name of your mail server. The syntax is simple -f = full -s = server -a = attributes -t = path and -z = compress.

Now comes a long waiting game, as you can see from this query this backup process took some 8+ hours to complete;

zmbackupquery -lb full-20091204.005207.351 -t /mnt/SANBackup/zimbra.backup/
Label: full-20091204.005207.351
Type: full
Status: completed
Started: Fri, 2009/12/04 10:52:07.351 EST
Ended: Fri, 2009/12/04 19:17:24.923 EST
Redo log sequence range: 3102 .. 3106
Number of accounts: 171

But of course since I like to cover myself I didn’t stop here I also decided to create an rsync copy of the mail store;

/sbin/rsync -avpHK /mnt/home/zimbra/ /mnt/SANBackup/MailRsync/

again go away for a few hours, then come back and hopefully you’ll have this;

du -h /mnt/home/zimbra/
167G /mnt/home/zimbra/

du -h /mnt/SANBackup/MailRsync/
167G /mnt/SANBackup/MailRsync/

Awesome we have two exact copies, the rsync syntax I use is a = archive v = verbose p = perseve permissions H = preserve hardlinks and K = keep directory links, ie treat a symlink as a directory.

Now that I felt we had a point to come back to if all went to hell it’s time to attack that partition. There are a couple of ways people will tell you to do this and the most commin would feather a Knoppix CD with gparted or similar. In the home use environment that would probably do the job and be less hands on but in this production environment it’s not an option and besides I am 200k’s from the server looking at a beach.

Will have to do the the console way and besides it’s quicker. There is a great GNU tool called parted and it’s what gparted is a front end for but in my experience it has issues with ext3 and journals, infact every time I’ve used it with ext3 I’ve gotten the error;

Error: Filesystem has incompatible feature enabled

to get around this you need to remove the file system features and we’ll do that later but for now I just suggest people forget about parted and use good ole fdisk

Of course I can hear the crys of “but fdisk can’t resize, only create and destroy” and you’d be right but we can use this to our advantage. See we don’t want to alter the starting cylinders just expand the partition to use more cylinders then before. Before we start you need to run;

/sbin/fdisk -l

Disk /dev/sdc: 429.4 GB, 429496729600 bytes
255 heads, 63 sectors/track, 52216 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdc1 1 26108 209712478+ 83 Linux

The reason we do this is to find our starting cylinder in my case it’s easy as I only have the 1 partition, so my starting cylinder is 1.. but if this partition was in the middle of the disk say you’d need to carefully write down this starting point because get it wrong later and you’re in for some pain as you’ll do damage to your data. From that output we can also see that my partition ends at 26108 but we want it to continue on till the end of the available space at 52216.

Here comes the fun part 🙂

First we need to stop all services in zimbra using the mail store;

zmcontrol stop
Stopping stats…Done
Stopping mta…Done
Stopping spell…Done
Stopping snmp…Done
Stopping archiving…Done
Stopping antivirus…Done
Stopping antispam…Done
Stopping imapproxy…Done
Stopping mailbox…Done
Stopping logger…Done
Stopping ldap…Done

you should just comfirm it really is all stopped with;

zmcontrol status
Unable to determine enabled services from ldap.
Enabled services read from cache. Service list may be inaccurate.
Host grange.langs.net.au
antispam Stopped
zmmtaconfigctl is not running
zmamavisdctl is not running
antivirus Stopped
zmmtaconfigctl is not running
zmamavisdctl is not running
zmclamdctl is not running
ldap Stopped
logger Stopped
logmysql.server is not running
zmlogswatchctl is not running
mailbox Stopped
zmmtaconfig is not running.
zmmtaconfigctl is not running
mysql.server is not running
zmconvertctl is not running
mailboxd is not running.
zmmailboxdctl is not running
mta Stopped
zmmtaconfigctl is not running
postfix is not running
zmsaslauthdctl is not running
snmp Stopped
zmswatch is not running.
spell Stopped
zmapachectl is not running
stats Stopped

and because I don’t have alot of faith I also run a script I came up with;

cat /opt/packages/killuser
#!/bin/bash
USER=$1
MYNAME=`basename $0`
if [ ! -n “$USER” ]
then
echo “Usage: $MYNAME username” >&2
exit 1
elif ! grep “^$USER:” /etc/passwd >/dev/null
then
echo “User $USER does not exist!” >&2
exit 2
fi
while [ `ps -ef | grep “^$USER” | wc -l` -gt 0 ]
do
PIDS=`ps -ef | grep “^$USER” | awk ‘{print $2}’`
echo “Killing ” `echo $PIDS | wc -w` ” processes for user $USER.”
for PID in $PIDS
do
kill -9 $PID 2>&1 >/dev/null
done
done
echo “User $USER has 0 processes still running.”

This will clean up any left over processes. Second part of this exercise is unmounting the file system the mail store uses;

umount /mnt/home

This has effectively parked our ext3 file system making it ready for manipulation.

We need to ensure the file system is in good order before we begin;

/sbin/fsck -n /dev/sdc1
fsck 1.35 (28-Feb-2004)
e2fsck 1.35 (28-Feb-2004)
/dev/sdc1: clean, 504961/26214400 files, 44472606/52428119 blocks

Okay looking good, now we need to remove those features that give parted a hard time by basically making our ext3 file system an ext2 file system, to do this we need to remove journaling;

/sbin/tune2fs -O ^has_journal /dev/sdc1
tune2fs 1.35 (28-Feb-2004)

Now it’s fdisk time, we’re going to delete the partition but don’t be alarmed because we’re not changing that starting cylinder remember and only going to expand the cylinder count this will be none destructive if you have claimly written down that starting cylinder and beside we took all those awesome backups;

[root@grange matthewd]# /sbin/fdisk /dev/sdc

The number of cylinders for this disk is set to 52216.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sdc: 429.4 GB, 429496729600 bytes
255 heads, 63 sectors/track, 52216 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdc1 1 26108 209712478+ 83 Linux

Command (m for help): d
Selected partition 1

Command (m for help): p

Disk /dev/sdc: 429.4 GB, 429496729600 bytes
255 heads, 63 sectors/track, 52216 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

At this point we’ve now removed the partition and it’s time to create the new one encompassing all the new space;

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-52216, default 1): 1
Last cylinder or +size or +sizeM or +sizeK (1-52216, default 52216): 52216

Command (m for help): p

Disk /dev/sdc: 429.4 GB, 429496729600 bytes
255 heads, 63 sectors/track, 52216 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdc1 1 52216 419424988+ 83 Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Right now it’s time for some husbandry on our new parition and file system;

/sbin/e2fsck -f /dev/sdc1
e2fsck 1.35 (28-Feb-2004)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/sdc1: 504961/26214400 files (25.1% non-contiguous), 44464404/52428119 blocks

Note this part takes some time so don’t be alarmed your data is safe, for me this was about an hour. Next do;

/sbin/resize2fs /dev/sdc1
resize2fs 1.35 (28-Feb-2004)
Resizing the filesystem on /dev/sdc1 to 104856247 (4k) blocks.
The filesystem on /dev/sdc1 is now 104856247 blocks long.

This is actually the step that grows your file system to now match the partition size. We need to check it one last time with;

/sbin/fsck -n /dev/sdc1
fsck 1.35 (28-Feb-2004)
e2fsck 1.35 (28-Feb-2004)
/dev/sdc1: clean, 504961/52428800 files, 45289879/104856247 blocks

Remember how we removed those file system features? Time to put them back getting our ext3 back in order;

/sbin/tune2fs -j /dev/sdc1
tune2fs 1.35 (28-Feb-2004)
Creating journal inode: mount done
This filesystem will be automatically checked every 20 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

Now lets remount our file system and see if a) the new size is there and b) that yes indeed we still have data;

mount /mnt/home/
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md0 178G 12G 157G 7% /
none 2.0G 0 2.0G 0% /dev/shm
pluto:/sanbackup 1.8T 871G 870G 51% /mnt/SANBackup
/dev/sdc1 394G 167G 212G 45% /mnt/home

Woot it’s mounted and our extra space is there.. now how about our data;

# ls -la /mnt/home
total 32
drwxr-xr-x 4 root root 4096 May 8 2008 .
drwxr-xr-x 4 root root 4096 May 8 2008 ..
drwx—— 2 root root 16384 Aug 1 2006 lost+found
drwxr-xr-x 4 zimbra zimbra 4096 Sep 13 2008 zimbra

# ls -la /mnt/home/zimbra
total 16
drwxr-xr-x 4 zimbra zimbra 4096 Sep 13 2008 .
drwxr-xr-x 4 root root 4096 May 8 2008 ..
drwxr-xr-x 3 zimbra zimbra 4096 Aug 29 2006 index
drwxr-xr-x 4 zimbra zimbra 4096 Aug 29 2006 store

All seems in order, so let s go ahead and restart our mail server;

zmcontrol start
Starting ldap…Done.
Starting logger…Done.
Starting mailbox…Done.
Starting antispam…Done.
Starting antivirus…Done.
Starting snmp…Done.
Starting spell…Done.
Starting mta…Done.
Starting stats…Done.

zmcontrol status
antispam Running
antivirus Running
ldap Running
logger Running
mailbox Running
mta Running
snmp Running
spell Running
stats Running

There we have it, mail system is back up and we no longer have to worry about the mail store running out of space and we didn’t lose anything so we even get to keep our job. A warning however do NOT try this with a windows box or if you are faint of heart, nor can I stress enough that you must always have the backups to go with it.

I promise you the day you don’t have a decent backup is the day you will need it, that’s just Murphy.