An ambitious star-mapping project highlights the growing importance of big data and the cloud, writes ARTHUR GOLDSTUCK.
At an event in Berlin today, the European Space Agency (ESA) is unveiling the biggest set of data about the stars ever gathered. The positions and magnitudes of no less than 1.7 billion stars of our Milky Way galaxy have been gathered by the Gaia spacecraft, which took off in 2013 and began collecting data a year later.
The ship is also transmitting a vast range of additional data, with distances, motions and colours of more than 1.3 billion stars collected so far. And that is without counting temperature measures, solar system analysis and radiation sources from outside the galaxy.
“The extraordinary data collected by Gaia throughout its mission will be used to eventually build the most accurate three-dimensional map of the positions, motions, and chemical composition of stars in our Galaxy,” according to a project document. “By reconstructing the properties and past trajectories of all the stars probed by Gaia, astronomers will be able to delve deep into the history of our Galaxy’s formation and evolution.”
The entire project would be impossible were it not for advances in cloud computing storage, big data analysis and artificial intelligence systems during this decade. The storage demands alone are mind-boggling. The ESA roped in cloud data services company NetApp, which focuses on management of applications and data across cloud and on-premise environments.
NetApp was previously involved with the Rosetta space mission, which landed a spacecraft on a comet in 2016. Lauched as far back as 2004, ten years later it became the first spacecraft to go into orbit around a comet, and its lander made the first successful landing on a comet.
“For the next two years Rosetta was following the comet and streaming data,” says Morne Bekker, NetApp South African country manager. “But with the comet speeding away from the sun at 120 000kph, Rosetta would soon lose solar power. Scientists seized the opportunity to attempt what no one had ever tried before — to gather unique observations through a controlled impact with the comet. Despite blistering speeds and countless unknowns, the spacecraft landed just 33m from its target point.
“It’s quite phenomenal when you think of the data and analytics harvested, and the information it can send back. Now we’re helping with the Gaia project. You can imagine how much data is being collected daily. The catalogue will probably end up at 2 Petabytes in size – that’s 2-million gigabytes. If you think of the minute points of data being extracted, obviously you have to be using AI and machine learning to analyse all of this.”
Ruben Alvarez, IT manager at the ESA, sums it up simply: “Data is everything. Our biggest challenge is processing of the data.”
Naturally, ESA required absolute reliability from data storage. It also demanded almost infinite scalability to support the massive data requirements of past, present, and future missions.
“We have a commitment to deliver data to different institutes in Europe on a daily basis,” says Alvarez. “Adding to the challenge, data from every mission must be accessible indefinitely. In the coming years, we will be launching new missions that will demand huge amounts of data. NetApp provided us with solutions that were scalable, even if we didn’t know in advance how much disk storage we were going to need.”
ESA says it expects to publish the full Gaia catalogue in 2020, making it available online to professional astronomers and the general public, with interactive, graphical interfaces.
The catalogue, says Alvarez, will unlock many mysteries of the stars.
“We call our site the Library of the Universe because we keep the science archive of
all of our scientific missions. This is how we allow people to really investigate the universe. t’s all about the data.”
The mission has tremendous scientific implications, but also makes a powerful business case for big data and cloud computing.
“The capabilities for AI and machine learning in the processing of mass amounts of data are far-reaching,” says Bekker. “Not only does it equate to extreme performance, but also to massive non-disruptive scalability where scientists can scale to 20 PB and beyond, to support the largest of learning data sets. Importantly it also allows scientists to expand their data where needed.”
Across Africa, the power of the cloud and big data is only slowly being harnessed. A new research project, Cloud Africa 2018, conducted by World Wide Worx for global networking application company F5 Networks, shows that cloud uptake is now pervasive across Kenya, Nigeria and South Africa.
However, the research reveals that each country experiences the benefits of the cloud differently. Respondents in Nigeria and Kenya named Business efficiency and Scalability by far the most important benefit, with 80% and 75% respectively selecting it as an advantage. Only 61% of South African respondents cited it.
The opposite happened with the most important benefit among South Africans: Time-to-market or speed of deployment came in as the most prominent, at 68% of respondents. In contrast, only 48% of companies in Kenya and 28% in Nigeria named it as a key benefit.
This appears to be a function of the infrastructure challenges in developing information technology markets like Nigeria and Kenya, where the cloud is used to overcome the obstacles that get in the way of efficiency.
In South Africa, where construction of the giant Square Kilometre Array multi radio telescope is due to begin next year, the learnings of Rosetta and Gaia will ensure that data collection, storage and analysis will no longer be a challenge.
- For the latest on project Gaia, visit http://sci.esa.int/gaia/
Prepare your cam to capture the Blood Moon
On 27 July 2018, South Africans can witness a total lunar eclipse, as the earth’s shadow completely covers the moon.
Also known as a blood or red moon, a total lunar eclipse is the most dramatic of all lunar eclipses and presents an exciting photographic opportunity for any aspiring photographer or would-be astronomers.
“A lunar eclipse is a rare cosmic sight. For centuries these events have inspired wonder, interest and sometimes fear amongst observers. Of course, if you are lucky to be around when one occurs, you would want to capture it all on camera,” says Dana Eitzen, Corporate and Marketing Communications Executive at Canon South Africa.
Canon ambassador and acclaimed landscape photographer David Noton has provided his top tips to keep in mind when photographing this occasion. In South Africa, the eclipse will be visible from about 19h14 on Friday, 27 July until 01h28 on the Saturday morning. The lunar eclipse will see the light from the sun blocked by the earth as it passes in front of the moon. The moon will turn red because of an effect known as Rayleigh Scattering, where bands of green and violet light become filtered through the atmosphere.
A partial eclipse will begin at 20h24 when the moon will start to turn red. The total eclipse begins at about 21h30 when the moon is completely red. The eclipse reaches its maximum at 22h21 when the moon is closest to the centre of the shadow.
David Noton advises:
- Download the right apps to be in-the-know
The sun’s position in the sky at any given time of day varies massively with latitude and season. That is not the case with the moon as its passage through the heavens is governed by its complex elliptical orbit of the earth. That orbit results in monthly, rather than seasonal variations, as the moon moves through its lunar cycle. The result is big differences in the timing of its appearance and its trajectory through the sky. Luckily, we no longer need to rely on weight tables to consult the behaviour of the moon, we can simply download an app on to our phone. The Photographer’s Ephemeris is useful for giving moonrise and moonset times, bearings and phases; while the Photopills app gives comprehensive information on the position of the moon in our sky. Armed with these two apps, I’m planning to shoot the Blood Moon rising in Dorset, England. I’m aiming to capture the moon within the first fifteen minutes of moonrise so I can catch it low in the sky and juxtapose it against an object on the horizon line for scale – this could be as simple as a tree on a hill.
- Invest in a lens with optimal zoom
On the 27th July, one of the key challenges we’ll face is shooting the moon large in the frame so we can see every crater on the asteroid pockmarked surface. It’s a task normally reserved for astronomers with super powerful telescopes, but if you’ve got a long telephoto lens on a full frame DSLR with around 600 mm of focal length, it can be done, depending on the composition. I will be using the Canon EOS 5D Mark IV with an EF 200-400mm f/4L IS USM Ext. 1.4 x lens.
- Use a tripod to capture the intimate details
As you frame up your shot, one thing will become immediately apparent; lunar tracking is incredibly challenging as the moon moves through the sky surprisingly quickly. As you’ll be using a long lens for this shoot, it’s important to invest in a sturdy tripod to help capture the best possible image. Although it will be tempting to take the shot by hand, it’s important to remember that your subject is over 384,000km away from you and even with a high shutter speed, the slightest of movements will become exaggerated.
- Integrate the moon into your landscape
Whilst images of the moon large in the frame can be beautifully detailed, they are essentially astronomical in their appeal. Personally, I’m far more drawn to using the lunar allure as an element in my landscapes, or using the moonlight as a light source. The latter is difficult, as the amount of light the moon reflects is tiny, whilst the lunar surface is so bright by comparison. Up to now, night photography meant long, long exposures but with cameras such as the Canon EOS-1D X Mark II and the Canon EOS 5D Mark IV now capable of astonishing low light performance, a whole new nocturnal world of opportunities has been opened to photographers.
- Master the shutter speed for your subject
The most evocative and genuine use of the moon in landscape portraits results from situations when the light on the moon balances with the twilight in the surrounding sky. Such images have a subtle appeal, mood and believability. By definition, any scene incorporating a medium or wide-angle view is going to render the moon as a tiny pin prick of light, but its presence will still be felt. Our eyes naturally gravitate to it, however insignificant it may seem. Of course, the issue of shutter speed is always there; too slow an exposure and all we’ll see is an unsightly lunar streak, even with a wide-angle lens.
On a clear night, mastering the shutter speed of your camera is integral to capturing the moon – exposing at 1/250 sec @ f8 ISO 100 (depending on focal length) is what you’ll need to stop the motion from blurring and if you are to get the technique right, with the high quality of cameras such as the Canon EOS 5DS R, you might even be able to see the twelve cameras that were left up there by NASA in the 60’s!
How Africa can embrace AI
Currently, no African country is among the top 10 countries expected to benefit most from AI and automation. But, the continent has the potential to catch up with the rest of world if we act fast, says ZOAIB HOOSEN, Microsoft Managing Director.
To play catch up, we must take advantage of our best and most powerful resource – our human capital. According to a report by the World Economic Forum (WEF), more than 60 percent of the population in sub-Saharan Africa is under the age of 25.
These are the people who are poised to create a future where humans and AI can work together for the good of society. In fact, the most recent WEF Global Shapers survey found that almost 80 percent of youth believe technology like AI is creating jobs rather than destroying them.
Staying ahead of the trends to stay employed
AI developments are expected to impact existing jobs, as AI can replicate certain activities at greater speed and scale. In some areas, AI could learn faster than humans, if not yet as deeply.
According to Gartner, while AI will improve the productivity of many jobs and create millions more new positions, it could impact many others. The simpler and less creative the job, the earlier, a bot for example, could replace it.
It’s important to stay ahead of the trends and find opportunities to expand our knowledge and skills while learning how to work more closely and symbiotically with technology.
Another global study by Accenture, found that the adoption of AI will create several new job categories requiring important and yet surprising skills. These include trainers, who are tasked with teaching AI systems how to perform; explainers, who bridge the gap between technologist and business leader; and sustainers, who ensure that AI systems are operating as designed.
It’s clear that successfully integrating human intelligence with AI, so they co-exist in a two-way learning relationship, will become more critical than ever.
Combining STEM with the arts
Young people have a leg up on those already in the working world because they can easily develop the necessary skills for these new roles. It’s therefore essential that our education system constantly evolves to equip youth with the right skills and way of thinking to be successful in jobs that may not even exist yet.
As the division of tasks between man and machine changes, we must re-evaluate the type of knowledge and skills imparted to future generations.
For example, technical skills will be required to design and implement AI systems, but interpersonal skills, creativity and emotional intelligence will also become crucial in giving humans an advantage over machines.
“At one level, AI will require that even more people specialise in digital skills and data science. But skilling-up for an AI-powered world involves more than science, technology, engineering and math. As computers behave more like humans, the social sciences and humanities will become even more important. Languages, art, history, economics, ethics, philosophy, psychology and human development courses can teach critical, philosophical and ethics-based skills that will be instrumental in the development and management of AI solutions.” This is according to Microsoft president, Brad Smith, and EVP of AI and research, Harry Shum, who recently authored the book “The Future Computed”, which primarily deals with AI and its role in society.
Interestingly, institutions like Stanford University are already implementing this forward-thinking approach. The university offers a programme called CS+X, which integrates its computer science degree with humanities degrees, resulting in a Bachelor of Arts and Science qualification.
Revisiting laws and regulation
For this type of evolution to happen, the onus is on policy makers to revisit current laws and even bring in new regulations. Policy makers need to identify the groups most at risk of losing their jobs and create strategies to reintegrate them into the economy.
Simultaneously, though AI could be hugely beneficial in areas such as curbing poor access to healthcare and improving diagnoses for example, physicians may avoid using this technology for fear of malpractice. To avoid this, we need regulation that closes the gap between the pace of technological change and that of regulatory response. It will also become essential to develop a code of ethics for this new ecosystem.
Preparing for the future
With the recent convergence of a transformative set of technologies, economies are entering a period in which AI has the potential overcome physical limitations and open up new sources of value and growth.
To avoid missing out on this opportunity, policy makers and business leaders must prepare for, and work toward, a future with AI. We must do so not with the idea that AI is simply another productivity enhancer. Rather, we must see AI as the tool that can transform our thinking about how growth is created.
It comes down to a choice of our people and economies being part of the technological disruption, or being left behind.