Nvidia’s New Maxine Platform is Phenomenal Applied AI Example

Nvidia just announced an innovative artificial intelligence application a couple of days ago and it’s big deal for a few reasons.

Let’s take a look at what it is and what it does first:

Friends doing teleconference with an open book on laptop

What is Nvidia Maxine?

Nvidia Maxine is an AI powered video streaming platform for developers. While it’s the platform that provides the technology that can be used for video calls or video conferencing, it’s not the end user product such as:

  • Skype, 
  • BlueJeans, 
  • Microsoft Teams, 
  • Google Duo, 
  • Hangouts,
  • Zoom 

It’s more the technology platform that applications could use if they wanted to.

AI became a chic word that’s used in so many products and/or services today but true AI innovation instantly shows off.

Traditionally, current video codecs (such as H.264) can be quite heavy on the internet connection due to its high quality and data size that comes with it. H.264 has been great for videos that are already recorded such as online movies, Netflix, Hulu, Youtube Videos etc. But when it comes to streaming it has the quality we want at a high internet speed cost.

Publicly trading video communications company Zoom has already seen a massive surge in stock prices going up to $485 in the spot market (as of October 7, 2020) from only approximately $70 in early January 2020 shortly before pandemic has broken out ZM:NASDAQ stock price on Google Finance.

What if we could have the support of predictive AI technology that made the application require 1/10 of the data size for a very similar image quality? That’s what Nvidia did.

Zoom shares price chart screenshot from Google Finance.

At this level Zoom’s market capitalization surpasses IBM’s approximate market cap of $110 billion US (IBM Market Capitalization Data on Nasdaq: IBM:NASDAQ) with $125 billion US.

Why is Nvidia Maxine special?

Nvidia’s Maxine implements Generative adversarial network, GANs, to create a deep fake of your own self.

Basically, what happens is, once the deep fake structure is created your movements, mimics, facial expressions, gestures and mannerism are transferred through the AI powered image that’s displayed to your counterparty. Apparently this allows massive amounts of data saving without overwhelming GPU or CPU too much.

We have already been seeing funny (or scary depending on the application and from where you look at) deep fake videos surface the internet recently. This is the immature phase of a technology when it first emerges and nobody is quite sure what to do with it.

Nvidia Maxine vs H.264 Demonstration Screenshot

I believe if Nvidia Maxine is mass adapted, which it definitely might especially if it can achieve its promises like 10x data saving, this will be the beginning of a new are where AI and particularly GAN are becoming mature end-user products and services.

Given the pandemic, more people than ever need this technology at affordable rates and 

Nvidia’s AI innovation couldn’t be timelier.

Given the public activities on Nvidia’s public Github repository and their extensive usage of Pytorch and Tensorflow for machine learning applications, Nvidia is highly likely to be using Python for their Maxine streaming platform as well.

Although Nvidia states the data size savings are up to 10x on the official webpage of Maxine here. In the promotion video Bandwidth usage per frame is announced as almost 900x ( 97KB vs 0.11KB ). There seems to be a confusion here but moving passed that, 10x data reduction without much quality loss is still huge deal.

On top of data savings Nvidia Maxine also promises to convert videos with 360p quality to 720p in a very realistic way. Here is the interesting promotion video from Nvidia AI Research:

Global Internet Usage

As of 2020, approximately 4.5 billion people have some form of internet access while 3.5 billion people still has no internet access whatsoever. 

According to UNESCO data this approximately corresponds to a ratio of 55% to  45% over the global population. This is absolutely a heart-breaking number considering how much we get done through the internet today both for business and pleasure.

During the pandemic times of Covid-19, for 100s of millions of people video conferencing has been a life saver (in addition to yoga, online shopping, dark chocolate, video games, video streaming services and remote work to name a few). Unfortunately, not all of us are equally blessed with the availability of tech at the tip of our fingers. 

On the flip side though, we have some good news. Internet usage stats will continue to get better and according to Cybersecurity Almanac (co-published by Cybersecurity Ventures and Cisco), global internet access percentage will increase to 6 billion in 2022 and all the way up to 7.5 billion in 2030 (90% of the projected population of 6+ yo ).

And, although 55% of the world seems to have access to internet, it doesn’t mean they all have high broadband connections. 6% of internet users in the US and 13% of internet users in Australia still has slow speed internet. (Reference: Weforum Internet Access Article)

Internet access statistics are approximately 9 in 10 in developed countries, 5 in 10 in developing countries and only barely 1 in 10 in underdeveloped countries.

Internet Logo in Front of Store

Also, when the internet is less available it tends to be very expensive and restricted and this can be a huge deal breaker for someone trying to put food on the table.

One thing these numbers expose is that video streaming, whether it’s for entertainment, online education (MOOCs) or video conferencing, is still nearly impossible for billions of people in 2020.

This statistical contrast makes it easier to comprehend the need of innovation in video streaming and to what extent it can change lives.

Future of Nvidia Maxine

It’s probably safe to conclude that this is just the beginning. Beginning of a very powerful and disruptive technology starting to creep into our lives.

Given the potential Maxine platforms enables, it’s probably safe to assume that its adaptation will be widespread very fast. We will watch and see how thing unfold for Nvidia’s Maxine platform.

Although Maxine already has the potential to make video conferencing available for millions of people with poor internet connection and save data at global scale we can see this application going further in terms of technological innovation.

Some of the applications Nvidia’s Maxine implementation can pioneer are:

  • VR
  • 3-D holograms
  • Online News
  • Presentations
  • MOOCs
  • Podcasts
  • Suitable Vlogs

On the flip side, this technology will inevitably also be used by people with malintent and we can see a surge in unpleasant experiences caused by:

  • online telemarketing
  • impersonation fraud 
  • more sophisticated scams
  • pranks
Let’s hope that our averaged out mutual conscious as the human kind continues improving as fast as technologies. We can also do more than hoping though. Best thing to do is familiarize and educate yourself and Holypython already offers a plethora of basic Python lessons, Python tutorials and Python exercises for online practice.
 
Techradar also has a pretty nice coverage on the news of Nvidia Maxine, you can find the Techradar article here.
 

You are also more than welcome to visit our recent Machine Learning section.

Code Comes to Arxiv

Open Knowledge for the People

We are very familiar with the open source movement as coders and its phenomenal success has been entirely transforming our planet.

In the last decade we have seen open source software activity skyrocket and even at commercial level open source alternatives have been replacing proprietary products with a close development environment. This is obviously the direction to go for computer science but we are now seeing more synergies than computer science.

Open Science works with the same philosophy and dynamics as Open Source, but it covers all things science. Another term Open Research or Open Scholarship are more common in social and human sciences as well as art domains.

Papers with Code

If you are interested in Machine Learning and AI, you might already know Papers with Code, an Open Machine Learning platform that hosts Machine Learning papers with code and evaluation tables, about page here.

This might seem as a no brainer to share Open Computer Science Research with a convenient showcase platform but as we are still de-learning our old ways of hoarding knowledge it took some time for more traditional domains and entities to follow suit.

If you are active or interested in Machine Learning definitely check out the papers shared on Papers with Code.

Papers With Code is an Existing Platform that Supports Open Machine Learning and Hosts ML Papers with Code

Arxiv.org

Arxiv on the other hand is a an open-access archive and free distribution service for scholarly articles and research results founded by Cornell University.

Common domains of edge-cutting papers that it hosts are:

  • physics,
  • mathematics,
  • computer science,
  • quantitative biology,
  • quantitative finance,
  • statistics,
  • electrical engineering
  • economics

If you check out Arxiv.org each field has very interesting sub-categories and specializations. For instance, these are the sub-topics of Computer Science:

  • Artificial Intelligence; 
  • Computation and Language; 
  • Computational Complexity; 
  • Computational Engineering, 
  • Finance and Science; 
  • Computational Geometry; 
  • Computer Science and Game Theory; 
  • Computer Vision and Pattern Recognition; 
  • Computers and Society; 
  • Cryptography and Security; 
  • Data Structures and Algorithms; 
  • Databases; 
  • Digital Libraries; 
  • Discrete Mathematics; 
  • Distributed, Parallel, and Cluster Computing; 
  • Emerging Technologies; 
  • Formal Languages and Automata Theory; 
  • General Literature; 
  • Graphics; 
  • Hardware Architecture; 
  • Human-Computer Interaction; 
  • Information Retrieval; 
  • Information Theory; 
  • Logic in Computer Science; 
  • Machine Learning; 
  • Mathematical Software; 
  • Multiagent Systems; 
  • Multimedia; 
  • Networking and Internet Architecture; 
  • Neural and Evolutionary Computing; 
  • Numerical Analysis; 
  • Operating Systems; 
  • Other Computer Science; 
  • Performance; 
  • Programming Languages; 
  • Robotics; 
  • Social and Information Networks; 
  • Software Engineering; 
  • Sound; 
  • Symbolic Computation; 
  • Systems and Control

Papers on Arxiv do go through some moderation and you have to be endorsed by an Arxiv publisher to be able to publish on Arxiv at first place. However, Arxiv papers are not peer-reviewed. This has some implications, which are not necessarily positive or negative:

  • Arxiv papers can be faster than traditional research papers since there isn’t the same amount of rigorous reviewing steps.
  • This also means they can have more mistakes than traditional papers
  • Arxiv papers are open to public for free. This means knowledge is shared freely without the paywalls people hit via traditional academia resources.
  • Arxiv papers get lots of public recognition and people can quickly start improving them, working on them or building on them.
  • Arxiv can be considered as a preliminary route before publishing through the traditional channels. Arxiv doesn’t cancel the opportunity for publishing a more complete research paper later but it rather enables a fast and quick feedback and free usage.

Arxiv with Code

Now, the news is that as of early October 2020, Arxiv implemented a code section in collaboration with “Papers with Code”. This means scientists and researchers can conveniently add the code involved in their paper directly on a Code Tab that’s added in the paper’s page.

This step might seem trivial to some people but it’s actually very big deal as it enables and encourages more and open sharing in terms of code.

This kind of practical innovation boosts overall activity and the quality of results in long term. We have been seeing it happen with open source work. Today, open source software reached a mind-blowing level.

For example, if you go to this qualifying paper: Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation, you can see that under the code tab its code is published and can be directly accessible through its Github link.

ample Paper Showing Official Code Repository Under New Arxiv Code Tab

This is incredible not only for CS or AI papers but papers in all fields and branches since coding is becoming a major enabler and amplifier of intellectual work. As more and more research is expected to include some form of code work in it this is a wise move from Cornell’s non-profit organization.

Conclusion

Arxiv’s move will not only make it easier to share research code but it will also make it easier to re-apply the work, experiment the same results, contribute to the existing work and come up with improvements in a more efficient fashion that hasn’t existed before.

Also, authors of papers could have been less reluctant to share the full code work due to restrictions before especially if the domain is not directly related to computer science or software development. This new feature can encourage all the code involved in a paper to be shared conveniently.

Let’s hope that Open Source philosophy will continue to influence more Open Science work and research from all aspects.

Made in Python: First Black Hole Image in History

The Event Horizon Telescope (EHT) is an impressive collaboration effort which created the first image of a black hole in history, solidifying Einstein’s general relativity theory.

Particular black hole that’s captured is at the center of Messier 87 galaxy, 55 million light years away and 6.5 times bigger than the mass of the sun.

Previously there has been attempts to capture black holes but observations were limited to jets of light coming from somewhere M87 black hole is suspected to be. (Hubble Faint Object Spectrograph was used to measure the rotation velocity of the ionized gas disk -astrophysical jet- at the center of M87 indicating a central black hole with 30% uncertainty.)

Now with the results EHT achieved, it’s official. 8 telescopes, 60 institutions across 20 countries contributed to this groundbreaking discovery. Telescopes used were: ALMAAPEXLarge Millimeter Telescope Alfonso SerranoJames Clerk Maxwell TelescopeIRAM 30-meter telescope, Submillimeter Array, Submillimeter Telescope, and South Pole Telescope.

South Pole Telescope

8 Telescopes synced their data with atomic clocks and each of them saved approximately 350TB of data per day stored on high-performance helium-filled hard drives. Run on 5 nights during a 10 night period in April 2017, 8 telescopes would generate approximately 14 Petabyte of data (1 Petabyte = 1000 Terabyte).

Drives were then flown by commercial planes to supercomputers: Max Planck Institute for Radio Astronomy and MIT Haystack Observatory and that’s where Python comes in.

AIPS is the standard data-reduction and analysis software package most commonly used for radio interferometers, including VLBI (Very-long-baseline interferometry).

Simulation Credit: Hotaka Shiokawa

First written in FORTRAN 66 in 1978, AIPS wasn’t the most suitable to work on EHT data since EHT data had its own characteristics. (Such as wide range of signal to noise, S/N, as opposed to ideal low S/N situation for AIPS)

Because of this scientists created an automated Python script based on AIPS. In addition to this custom pipeline, several standard Python libraries like scipy, pylab, matplotlib, and numpy were utilized during the project.

Black holes are one of the major frontiers in physics and our understanding of universe. Since all the observations in the project are consistent with Einstein’s theory, it also suggests that intense gravity provided by a black hole might be bending spacetime and creating two end points in spacetime.

First black hole image captured ever. (Stitched together in Python)

New millenia became a playground for all Sci-fi plots to come alive and it probably doesn’t get more Sci-fi than wormholes becoming an observed phenomenon.

You can find the Python custom AIPS pipeline on Dr. Intema’s webpage here.
Event Horizon Telescope’s Github repository can be found here.

Your code is going on an Arctic journey soon.

Norway has an island called Svalbard which is full of polar bears. It’s actually an archipelago or a group of islands if you will and also home to the Northern most town in the world.

In the last decades Svalbard has been a host to numerous Hollywood movies due to the arctic feeling and polar bears. Orion’s Belt, Die Another Day, The Golden Compass, Eight Below, Frozen Planet just to name a few.

In year 2008, Svalbard Global Seed Vault, has been launched to create a space for preserving seed samples from all around the world, just in case.

Global Seed Vault

Now, GitHub is doing a similar project, this time with the open-source codes of millions of developers on Github.

Github announced that they have partnered with Arctic World Archive which partnered with an old decommissioned mine in Svalbard which goes 250 meter deep to preserve the codes with the use of films.

AWA states it mission as "to help preserve the world's digital memory and ensure that the world's most irreplaceable digital memories of art, culture and literature are secured and made available to future generations" on their cool website.

It makes sense to preserve all the public repositories we all depend on in case of a Force Majeure that’s big enough to wipe out majority of our civilization on planet Earth.

Piql AS is the company behind the durable film technology that will be used for storing the codes for the next 1000 years.

GitHub has already deposited 6,000 of its most significant repositories in AWA, capturing the evolution of technology and software. This collection includes source codes for some of the most significant repositories such as: Linux and Android operating systems; Python, Ruby, Rust programming languages; Node, V8, React, and Angular; Bitcoin and Ethereum; TensorFlow and FastAI; and many more. With Arctic Code Vault project, GitHub will now store all active public repositories.

Arctic Code Vault project will take place on February 2, 2020 so if your public repositories have bugs in them you might want to clean up before your program freezes for the next 1000 years, literally.

Python praise of a Nobel Prize winner economist

Paul Romer is the embodiment of academic success. And in 2018 he came under the global spotlight by winning the Nobel Prize for Economics for his contributions to the understanding of long-term economic growth and its relation to technological innovation.

He was the Chief Economist and Senior Vice President of the World Bank and professor of economics at NYU, the University of Chicago, the University of California, Berkeley, Stanford University and the University of Rochester.

That’s why it made news when he praised Python and Jupyter Notebook in a long article saying all kind of nice things about the open source programming language and browser based development environment.

It’s often not a good idea to bet against a Nobel prize winner and also in this case it’s hard to disagree with any of the points he made about Python and Jupyter especially since Paul Romer’s Nobel prize was about technological innovation’s relation with long-term economic growth.

Here are 10 points parallel to what he wrote about Python and Jupyter:

 

    1. Open Source dominates.


      Unlike Mathematica, Matlab and Excel, Jupyter and Python are open source programs. It means you can see exactly what’s under the hood anytime you’d like. It’s all open source and out there for the public to view, review, examine, admire and get inspired by. This is a very different story than proprietary software which is a black box with no sneak peak.

      “It is along this social dimension that open source unambiguously dominates the proprietary model. Moreover, at a time when trust and truth are in retreat, the social dimension is the one that matters.”

    2. Transparency over secrecy.

      Professor Romer suggests in his post that: “Jupyter rewards transparency; Mathematica rationalizes secrecy. Jupyter encourages individual integrity; Mathematica lets individuals hide behind corporate evasion. Jupyter exemplifies the social systems that emerged from the Scientific Revolution and the Enlightenment, systems that make it possible for people to cooperate by committing to objective truth; Mathematica exemplifies the horde of new Vandals whose pursuit of private gain threatens a far greater pubic loss–the collapse of social systems that took centuries to build.”
      Strong words indeed.

    3. Easy to use.

      He goes on: “It [Jupyter] offers the best REPL I’ve ever used. It lets me get quick feedback, via text or graphics, about what happens when I select a line of code and run it.”

      This sums up the amazing convenience of Jupyter in research and education very nicely.

    4. Excellent Libraries

      This one should be obvious, Python has some of the best -and most abundant- libraries in the programming world: “Python libraries let me replicate everything I wanted to do with Mathematica: Matplotlib for graphics, SymPy for symbolic math, NumPy and SciPy for numerical calculations, Pandas for data, and NLTK for natural language processing.”

    5. Awesome Community.

      “I’m more productive. I’m having fun. On both counts, it helps to be able to get an honest answer when I have a question.”

You can read his original April 2018 blog post here.

Paul Romer

Inspirational Quotes by Paul Romer

Paul Romer has also said or published some thought provoking quotes about innovation, tech and economics before. Here is a curation of 4 quotes he said. If you’d like to read more about Professor Romer or his work you can check out this book on Amazon written about him by author David Warsh below or you can read his blog here.
Quotes

  1. Every generation has underestimated the potential for finding new ideas . . . Possibilities do not add up. They multiply.
  2. Growth springs from better recipes, not just from more cooking.
  3. Human material existence is limited by ideas, not stuff, people don’t need copper wires they need ways to communicate, oil was a contaminant, then it became a fuel
  4. Never waste a crisis.