Thanks to the political process, Mattis said America is now looking towards victory in Afghanistan after more than 16 years of conflict.
The visit comes two weeks after President Ghani invited the Taliban to begin peace talks without preconditions to end the 16-year-old war in the Kabul Process Conference last month.
Mattis said the goal was to take advantage of Taliban fracturing to peel off insurgents exhausted of fighting and create a process that could incorporate some Taliban leaders.
The group said last month that it was open to reaching a political settlement and negotiating, but has not responded to President Ghani's peace offer so far.
United Nations said that over 445,000 people were displaced due to conflict in the country in 2017.
Afghan security forces recaptured a district headquarters in western Farah province on Monday, just hours after the Taliban overrun the police and administrative offices, killing eight policemen, a spokesman said.
The U.S. has a renewed focus on Afghanistan after years of drawdowns under former president Barack Obama and talk by top U.S. generals of "not winning" and of a "stalemate" in the seemingly intractable conflict.
"Right now the district headquarters and the area around it are under control of the Afghan security forces", Rahimi said, adding that "fighting is ongoing in three different parts of the district" as Afghan forces pursue the Taliban.
He will also meet U.S. commanders and allied troops posted in the country, TOLO news reported.
"We've had some groups of Taliban - small groups - who have either started to come over or expressed an interest in talking", Mattis said.
The U.S. has increased its military presence and activities in Afghanistan under a more aggressive policy ordered by President Donald Trump, boosting the number of forces from 8,500 to about 14,000.
They have offered to talk directly to the United States about a possible peace agreement.
Mattis said he had seen some positive indications from Islamabad, noting some Pakistani military operations along the border.
"Afghan security forces have been stretched thin as fighting escalated after winter ended, " the United Nations said.
Mattis previously visited Afghanistan in April and September 2017.