An ambitious star-mapping project highlights the growing importance of big data and the cloud, writes ARTHUR GOLDSTUCK.
At an event in Berlin today, the European Space Agency (ESA) is unveiling the biggest set of data about the stars ever gathered. The positions and magnitudes of no less than 1.7 billion stars of our Milky Way galaxy have been gathered by the Gaia spacecraft, which took off in 2013 and began collecting data a year later.
The ship is also transmitting a vast range of additional data, with distances, motions and colours of more than 1.3 billion stars collected so far. And that is without counting temperature measures, solar system analysis and radiation sources from outside the galaxy.
“The extraordinary data collected by Gaia throughout its mission will be used to eventually build the most accurate three-dimensional map of the positions, motions, and chemical composition of stars in our Galaxy,” according to a project document. “By reconstructing the properties and past trajectories of all the stars probed by Gaia, astronomers will be able to delve deep into the history of our Galaxy’s formation and evolution.”
The entire project would be impossible were it not for advances in cloud computing storage, big data analysis and artificial intelligence systems during this decade. The storage demands alone are mind-boggling. The ESA roped in cloud data services company NetApp, which focuses on management of applications and data across cloud and on-premise environments.
NetApp was previously involved with the Rosetta space mission, which landed a spacecraft on a comet in 2016. Lauched as far back as 2004, ten years later it became the first spacecraft to go into orbit around a comet, and its lander made the first successful landing on a comet.
“For the next two years Rosetta was following the comet and streaming data,” says Morne Bekker, NetApp South African country manager. “But with the comet speeding away from the sun at 120 000kph, Rosetta would soon lose solar power. Scientists seized the opportunity to attempt what no one had ever tried before — to gather unique observations through a controlled impact with the comet. Despite blistering speeds and countless unknowns, the spacecraft landed just 33m from its target point.
“It’s quite phenomenal when you think of the data and analytics harvested, and the information it can send back. Now we’re helping with the Gaia project. You can imagine how much data is being collected daily. The catalogue will probably end up at 2 Petabytes in size – that’s 2-million gigabytes. If you think of the minute points of data being extracted, obviously you have to be using AI and machine learning to analyse all of this.”
Ruben Alvarez, IT manager at the ESA, sums it up simply: “Data is everything. Our biggest challenge is processing of the data.”
Naturally, ESA required absolute reliability from data storage. It also demanded almost infinite scalability to support the massive data requirements of past, present, and future missions.
“We have a commitment to deliver data to different institutes in Europe on a daily basis,” says Alvarez. “Adding to the challenge, data from every mission must be accessible indefinitely. In the coming years, we will be launching new missions that will demand huge amounts of data. NetApp provided us with solutions that were scalable, even if we didn’t know in advance how much disk storage we were going to need.”
ESA says it expects to publish the full Gaia catalogue in 2020, making it available online to professional astronomers and the general public, with interactive, graphical interfaces.
The catalogue, says Alvarez, will unlock many mysteries of the stars.
“We call our site the Library of the Universe because we keep the science archive of
all of our scientific missions. This is how we allow people to really investigate the universe. t’s all about the data.”
The mission has tremendous scientific implications, but also makes a powerful business case for big data and cloud computing.
“The capabilities for AI and machine learning in the processing of mass amounts of data are far-reaching,” says Bekker. “Not only does it equate to extreme performance, but also to massive non-disruptive scalability where scientists can scale to 20 PB and beyond, to support the largest of learning data sets. Importantly it also allows scientists to expand their data where needed.”
Across Africa, the power of the cloud and big data is only slowly being harnessed. A new research project, Cloud Africa 2018, conducted by World Wide Worx for global networking application company F5 Networks, shows that cloud uptake is now pervasive across Kenya, Nigeria and South Africa.
However, the research reveals that each country experiences the benefits of the cloud differently. Respondents in Nigeria and Kenya named Business efficiency and Scalability by far the most important benefit, with 80% and 75% respectively selecting it as an advantage. Only 61% of South African respondents cited it.
The opposite happened with the most important benefit among South Africans: Time-to-market or speed of deployment came in as the most prominent, at 68% of respondents. In contrast, only 48% of companies in Kenya and 28% in Nigeria named it as a key benefit.
This appears to be a function of the infrastructure challenges in developing information technology markets like Nigeria and Kenya, where the cloud is used to overcome the obstacles that get in the way of efficiency.
In South Africa, where construction of the giant Square Kilometre Array multi radio telescope is due to begin next year, the learnings of Rosetta and Gaia will ensure that data collection, storage and analysis will no longer be a challenge.
- For the latest on project Gaia, visit http://sci.esa.int/gaia/
Money talks and electronic gaming evolves
Computer gaming has evolved dramatically in the last two years, as it follows the money, writes ARTHUR GOLDSTUCK in the second of a two-part series.
The clue that gaming has become big business in South Africa was delivered by a non-gaming brand. When Comic Con, an American popular culture convention that has become a mecca for comics enthusiasts, was hosted in South Arica for the first time last month, it used gaming as the major drawcard. More than 45 000 people attended.
The event and its attendance was expected to be a major dampener for the annual rAge gaming expo, which took place just weeks later. Instead, rAge saw only a marginal fall in visitor numbers. No less than 34 000 people descended on the Ticketpro Dome for the chaos of cosplay, LAN gaming, virtual reality, board gaming and new video games.
It proved not only that there was room for more than one major gaming event, but also that a massive market exists for the sector in South Africa. And with a large market, one also found numerous gaming niches that either emerged afresh or will keep going over the years. One of these, LAN (for Local Area Network) gaming, which sees hordes of players camping out at the venue for three days to play each other on elaborate computer rigs, was back as strong as ever at rAge.
MWeb provided an 8Gbps line to the expo, to connect all these gamers, and recorded 120TB in downloads and 15Tb in uploads – a total that would have used up the entire country’s bandwidth a few years ago.
“LANs are supposed to be a thing of the past, yet we buck the trend each year,” says Michael James, senior project manager and owner of rAge. “It is more of a spectacle than a simple LAN, so I can understand.”
New phenomena, often associated with the flavour of the moment, also emerge every year.
“Fortnite is a good example this year of how we evolve,” says James. “It’s a crazy huge phenomenon and nobody was servicing the demand from a tournament point of view. So rAge and Xbox created a casual LAN tournament that anyone could enter and win a prize. I think the top 10 people got something each round.”
Read on to see how esports is starting to make an impact in gaming.
Blockchain is generally associated with Bitcoin and other cryptocurrencies, but these are just the tip of the iceberg, says ESET Southern Africa.
This technology was originally conceived in 1991, when Stuart Haber and W. Scott Stornetta described their first work on a chain of cryptographically secured blocks, but only gained notoriety in 2008, when it became popular with the arrival of Bitcoin. It is currently gaining demand in other commercial applications and its annual growth is expected to reach 51% by 2022 in numerous markets, such as those of financial institutions and the Internet of Things (IoT), according to MarketWatch.
What is blockchain?
A blockchain is a unique, consensual record that is distributed over multiple network nodes. In the case of cryptocurrencies, think of it as the accounting ledger where each transaction is recorded.
A blockchain transaction is complex and can be difficult to understand if you delve into the inner details of how it works, but the basic idea is simple to follow.
Each block stores:
– A number of valid records or transactions.
– Information referring to that block.
– A link to the previous block and next block through the hash of each block—a unique code that can be thought of as the block’s fingerprint.
Accordingly, each block has a specific and immovable place within the chain, since each block contains information from the hash of the previous block. The entire chain is stored in each network node that makes up the blockchain, so an exact copy of the chain is stored in all network participants.
As new records are created, they are first verified and validated by the network nodes and then added to a new block that is linked to the chain.
How is blockchain so secure?
Being a distributed technology in which each network node stores an exact copy of the chain, the availability of the information is guaranteed at all times. So if an attacker wanted to cause a denial-of-service attack, they would have to annul all network nodes since it only takes one node to be operative for the information to be available.
Besides that, since each record is consensual, and all nodes contain the same information, it is almost impossible to alter it, ensuring its integrity. If an attacker wanted to modify the information in a blockchain, they would have to modify the entire chain in at least 51% of the nodes.
In blockchain, data is distributed across all network nodes. With no central node, all participate equally, storing, and validating all information. It is a very powerful tool for transmitting and storing information in a reliable way; a decentralised model in which the information belongs to us, since we do not need a company to provide the service.
What else can blockchain be used for?
Essentially, blockchain can be used to store any type of information that must be kept intact and remain available in a secure, decentralised and cheaper way than through intermediaries. Moreover, since the information stored is encrypted, its confidentiality can be guaranteed, as only those who have the encryption key can access it.
Use of blockchain in healthcare
Health records could be consolidated and stored in blockchain, for instance. This would mean that the medical history of each patient would be safe and, at the same time, available to each doctor authorised, regardless of the health centre where the patient was treated. Even the pharmaceutical industry could use this technology to verify medicines and prevent counterfeiting.
Use of blockchain for documents
Blockchain would also be very useful for managing digital assets and documentation. Up to now, the problem with digital is that everything is easy to copy, but Blockchain allows you to record purchases, deeds, documents, or any other type of online asset without them being falsified.
Other blockchain uses
This technology could also revolutionise the Internet of Things (IoT) market where the challenge lies in the millions of devices connected to the internet that must be managed by the supplier companies. In a few years’ time, the centralised model won’t be able to support so many devices, not to mention the fact that many of these are not secure enough. With blockchain, devices can communicate through the network directly, safely, and reliably with no need for intermediaries.
Blockchain allows you to verify, validate, track, and store all types of information, from digital certificates, democratic voting systems, logistics and messaging services, to intelligent contracts and, of course, money and financial transactions.
Without doubt, blockchain has turned the immutable and decentralized layer the internet has always dreamed about into a reality. This technology takes reliance out of the equation and replaces it with mathematical fact.