Mr Biden said he thought of Mr McCain as a brother "with a lot of family fights".
Many might wonder what young African-American kid from Minnesota, and a highly decorated war hero turned United States Senator might have in common, he said.
Family members watched in silence as uniformed military members removed the flag-draped casket from a black hearse and carried it into the church.
McCain passed away Saturday at the age of 81 after a yearlong battle with an aggressive brain cancer. The hymn "Amazing Grace" is to be performed by the Brophy Student Ensemble, from the college preparatory school in Phoenix attended by McCain's sons, Jack and Jimmy.
McCain's longtime chief of staff Grant Woods, a former Arizona attorney general, drew laughs with a eulogy in which he talked about McCain's "terribly bad driving" and his sense of humor, which included calling the Leisure World retirement community "Seizure World".
The Latest on services for Sen.
"On behalf of a grateful nation, we will ever remember that John McCain served his country and John McCain served his country honorably'". (Sarah Silbiger/CQ Roll Call) Members of the different services were bearers of McCain's casket into the Capitol.
"He was a great fire who burned bright, and we lived in his light and warmth for so very long", she continued.
Dabbing his eyes as he recalled his friend, Biden said McCain "could not stand the abuse of power wherever he saw it, in whatever form, in whatever country". Biden is a Democrat whose friendship endured an election that pitted them against one another. Drawing on the tragedies suffered in his own life - the deaths of his first wife and daughter in a auto accident in 1972, and his son Beau from cancer in 2015 - Biden offered consolation to McCain's family, calling his absence "all-consuming". Fitzgerald last visited McCain and his wife Cindy at their home near Sedona in February.
Touching on his own losses, of his first wife and daughter, who died in a vehicle accident in 1972, and his son Beau in 2015, he counseled the McCain family on how to soldier on through their grief, and compared loss to "being sucked into a black hole inside your chest".
At the end of the almost 90-minute ceremony, McCain's casket was wheeled out of the church to "My Way", in tribute to a politician known for following his own path.
Pence said that President Donald Trump - a man who was not invited to services for McCain - asked him to be there.
Sen. Tammy Duckworth, an Illinois Democrat, told reporters of McCain "We need people with his values here".
After Thursday's church service, a military aircraft was scheduled to take McCain's body back east for a lying-in-state at the U.S. Capitol on Friday, a service at the Washington National Cathedral on Saturday, and burial at the U.S. Naval Academy in Annapolis, Maryland.
As the 11-vehicle motorcade with a 17-motorcycle police escort made its way along the 8-mile (13-kilometer) route, people held signs that read simply "McCain, " and cars on the other side of the highway stopped or slowed to a crawl in apparent tribute. President Trump, who had publicly feuded with McCain, is not expected to attend.
A portrait and a birthday balloon are left outside the Arizona State Capitol on Wednesday, which would have been McCain's 82nd birthday. Meghan McCain, his daughter, leaned into the casket and wept.
- Thousands of admirers of the late Senator John McCain stood in line for hours on Wednesday in the blazing Arizona sun and triple-digit heat for a chance to pay final respects to the war hero and two-time Republican presidential candidate.
Michael Fellars was among those awaiting the motorcade outside the church Thursday.
"He cared about the substance of my heart moreso than where I came from", he said.