theartsdesk in Sydney: Upside Down Under

Sydney has a nervous tic. People think Australians are brash and bolshy but that's not true. There's a deep sense of ingrained anxiety here. That anxiety comes from being at the edge of the world, a long way from Europe and in an unfamiliar and unrelenting land. It has been expressed through the art of Australia for 200 years. Today the country and its biggest city are both more confident, so the anxiety expresses itself in subtler ways.

Yet Sydney still cares what people think. Jorn Utzon's Sydney Opera House presides over the Harbour and in its 40th year it is still an absolute bobby dazzler. But the whole point of the building was to prove that Sydney and ergo Australia wasn't “Terra Nullius” – the empty land – at all. The whole point of the building was to prove that this was a cultured city in a cultured country. Sydney Festival tries to do that same thing today.

Inside Utzon's Opera House I watch Black Diggers, a play by Tom Wright and directed by Wesley Enoch which is a very interesting little thing indeed. Because, for all the white anxiety about this land and latterly about its image abroad, the reality of life for the people we, as colonial Britons, dispossessed in the 1800s and 1900s was very much worse than the nightmares Europeans had after dark, fuelled by the sounds of grouchy kookaburras and exuberant possums. Indigenous Australians were second-class citizens. Their nightmares were real. What has happened to them is a disgrace. Which makes it all the more fascinating that so many brave young "black diggers" – 1000 of them – signed up to fight for the Australian and New Zealand Army Corps in a European war which wasn't going to win freedom for them or their families back home.

For all that self-consciousness, Aussies do love to celebrate

This tightly coiled drama effectively unpicks this secret history in the centenary year for World War One and full marks to it for doing so. It's caused articles to be written around the world, and on the back of John Pilger's recent and traumatic new film Utopia, asks us to think again about what we did, what we were complicit in. The audience is prodded and provoked in this play and so we should be. This is what drama is for and this is what arts festivals are capable of.

Australian art has taken something of a kicking of late, what with the lukewarm reviews of the recent Australian art exhibition at the Royal Academy in London. But Sydney is a city where art is really cherished. It probably hurt Australians a fair bit to see such slatings and it feeds into that nagging sense that their own cultural offerings aren't always up to scratch. The art at Sydney Festival is well programmed and interesting in the main.

Christian Boltanski's Chance seems almost ideally partnered to Black Diggers because its central concern is also death and the trauma of memory; the trauma of dead relatives is such a powerful force for indigenous Australians that they can't look at images of dead people on TV – warnings have to be broadcast. Western society seems to have become blinded to death, we treat it without respect. Bolstanski, a Parisian, asks us to question that – his videos of faces and bodies sliding across digital screens anchored to steel scaffolding at the Carriageworks are as provocative as the Black Diggers script. Locals don't notice the location so much, but I do: the Carriageworks is the former Eveleigh Rail Yards, dating back to the 19th century. These huge industrial sheds provided work for the people who lived here but also reinforced colonial power – railways were British, industry was British, the best carriages for the Governor General and for visiting royals were built here. It's an eerie and exciting place.

Down in Darling Harbour, surrounded by the po-mo kitsch architecture of this odd 1980s development, Leandro Erlich's Merchant Store offers more visceral and certainly uplifting thrills. Who can fail to be excited about climbing up a building, like Spiderman? Well, crawling along a building on the floor actually, for Erlich's art is all smoke and mirrors – one giant one, tilted, means that a visual illusion is created which everyone seems to love. Accessible art doesn't come more enjoyable than this and the trick is a similar one to that which Erlich pulled off with Dalston House in East London earlier this year. But as with Black Diggers and Chance, history weighs heavy – Erlich could've chosen any building in Sydney but he chose the 19th-century Union Bond Store in George Street as the basis for his recreation, which means that colonialism seems to just keep coming back and biting festival organisers, and Sydneysiders in general, on the arse, no matter how much they may dream of forgetting and looking forward.

But at least Jeremy Deller hasn't gone and plonked down a huge art work in the middle of the city which references the motherland. Oh wait... he has. Deller's inflatable Stonehenge replica, called Sacrilege, crowns Hyde Park (Sydney's, not London's) and again provides a ton of free thrills. But if there's one thing which is just about the art and not about the past, then Florentijn Hofma's giant, inflatable yellow duck is it. It's just a big friendly yellow duck. No hidden meanings here – well not that I could fathom after staring at it for 20 minutes anyway.

The festival itself is a fun affair and even in the midst of these serious and important works – and the more lighthearted ones of course – there's a feeling of celebration. Because of course, for all that self-consciousness, Aussies do love to celebrate. The nocturnal feel of the main festival site in Hyde Park epitomises this, with a warm and friendly atmosphere. The circus tent here hosts a compellingly weird and intimate Cat Power performance. A closing set from Andrew Weatherall tops off the music part of the festival.

Sydney, of course, is paradise – the first inhabitants of this city realised it, and eventually the rather less keenly settled second generation in chains came to realise it too. Perhaps in the future we can all share space and culture. Perhaps future Sydney Festivals will offer more from the the very first inhabitants and less from the second. Anyway, it's a heaven on earth and that cultural uncertainty – well, there's no need to fret on it. Better to be cautious than cocky like so much of the London arts scene.

• The next Sydney Festival is in Jan 2015
• Christopher Beanland was a guest of New South Wales Tourism and Qantas