Imagine 100 million divers jumping into the ocean at once.
Your task: Using high speed cameras, photograph the wake of each splash, then determine and plot exactly where each diver entered the water.
That’s somewhat analogous to what a group of NIU physicists were trying to accomplish earlier this year using a technique known as “speckle visibility spectroscopy” at Argonne National Laboratory’s Advanced Photon Source (APS).
The APS facility produces the brightest storage ring-generated X-ray beams in the Western Hemisphere, enabling researchers to peer into the molecular world and gather enormous amounts of data.
The sheer data volume would overwhelm most computer systems, but the NIU scientists would end up turning to a relatively new computing resource on campus, with remarkable results.
At Argonne, the physicists, including Ph.D. student Nuwan Karunaratne and NIU physics chair Larry Lurio, were testing a way to study the diffusion, or intermingling, of a certain type of proteins called crystallins.
Crystallins make up the lens of the eye. Over time, these proteins can get damaged and start to stick to each other, leading to a disease known as presbyopia, in which the lens of the eye becomes rigid and loses its ability to focus.
“We are trying to see how aging affects the eye-lens flexibility at the molecular level, and this will require that we can see the interaction of individual proteins,” Lurio says. “Chemistry and biology professor Elizabeth Gaillard is doing applied work in this area in her studies of eye disease. Although the projects are separate, the research is synergistic, and we’ve learned a lot from Dr. Gaillard’s work.”
Argonne’s super X-ray can capture individual molecules or proteins, and high speed cameras can help pinpoint their positions. But taking a series of high speed photos, and then determining and documenting each protein’s movement over time, is a mighty task requiring large amounts of computing power.
For the physicists’ “proof of principle” experiment, they studied latex particles, which are the same size as proteins, over a week-long period at the APS. In the past, Lurio says, NIU scientists would book coveted time at the APS, conduct their experiments and then take data back to the university to be run through computers and analyzed.
If the researchers wanted to focus on something especially interesting that occurred during the experiment, they would have to return to the APS at a later time. On the other hand, if for some reason an experiment wasn’t working correctly, the scientists wouldn’t learn about it until much later. The whole experiment could be a bust.
Enter Gaea, NIU’s hybrid GPU/CPU supercomputer, or cluster of computers, which came online in 2012. It has a capacity of more than 30 teraflops, meaning it can do more than 30 trillion calculations per second.
Scientists and students from NIU’s Department of Computer Science worked with physicists during their APS experiments. Using Gaea, which can be accessed remotely, they achieved near real-time results.
“In the past, we’d take home huge reams of data and it would take a month to analyze, but when we did this experiment in February, we could see the results as they came in,” Lurio says. “It’s amazing and very beneficial. The time scientists have to use the facilities at Argonne is precious. With real-time feedback, you can decide if a sample is good or whether you need to make adjustments.”
NIU’s hybrid GPU/CPU supercomputer is expected to usher in a new era of high performance computing at the university, opening up opportunities for faculty researchers in a wide of array disciplines and for students interested in supercomputing.
The cluster’s use of both Central Processing Units (CPUs) and Graphics Processing Units (GPUs) is an important feature. Working in tandem, they pack a powerful computational punch.
John Winans, a research associate in computer science, manages the department’s High Performance Computing Laboratory, which includes the supercomputing cluster. Currently, three undergraduates and two graduate students also work with the cluster, providing support to researchers.
“All of them are working on research projects with faculty from across campus, Fermilab and Argonne,” Winans says.
Staff members from the university’s Information Technology Services also have been known to lend their expertise, along with Computer Science Professor Kirk Duffin and Nicholas Karonis, a supercomputing expert and NIU computer science chair.
“Nick is extremely busy, but we needed his assistance on our project, so we locked him in a room for five or six hours so he could write code,” Lurio says. “Then he popped out with a solution.
“Having trained computer scientists who can collaborate with the researchers is as valuable as the computer cluster itself,” Lurio adds.
Karonis says the cluster is being used on about 18 research projects on and off campus, ranging from bioinformatics to detector physics.
“For Larry and his group, it was a game changer,” Karonis says. “The Gaea cluster allowed them to produce results in minutes, instead of weeks.
“We know it can be a game changer for other researchers across campus as well,” Karonis adds. He notes that the services are free, thanks to the combined support through external funding from Professor Emeritus Clyde Kimball and from the Office of the Vice President for Research and Graduate Studies.
Faculty members who are interested in potentially taking advantage of the Gaea cluster for their research projects are encouraged to contact Karonis via email at firstname.lastname@example.org.
“We’ll assess whether Gaea’s computing power can help with your research,” Karonis says. “If desired, we can even design a path that will take the project onto the cluster and write code for its implementation.”
Argonne scientists Alec Sandy and Suresh Narayanan were key collaborators in the APS experiment, along with George Thurston, a physics professor at the Rochester Institute of Technology.