One reciting the Amidah must have kavanah; that is, he must focus on what he is saying, and he must try not to let his mind be distracted by anything else during the prayer. If other thoughts enter his mind, he must expel them and return to his prayer. Even if he does not succeed in concentrating on all of the words, he must at least try to have kavanah for the conclusion of each berachah. If he cannot concentrate during all of the berachot, he must make an effort to concentrate in Birkat Avot and Birkat Modim, for those are the berachot in which we bow down at their beginning and at their end. At the very least, he must have kavanah in Birkat Avot, the berachah that opens the Amidah.
If a person recited the Amidah and did not have kavanah during Birkat Avot, he must, in principle, repeat his Amidah, because the lack of kavanah during that berachah prevents him from fulfilling his obligation. However, as a result of the deterioration of the generations and troubles on the mind, our ability to concentrate has weakened. Therefore, the Acharonim have instructed not to repeat the Amidah, since it is possible that even in reciting the Amidah a second time, he will forget to have kavanah in Birkat Avot and his repetition will be for naught (Rama 101:1; Kaf HaChaim 4).
A person who is about to finish Birkat Avot and notices that he did not have kavanah in its recital: as long as he has not yet said Hashem’s Name at the conclusion of the berachah, he goes back to “Elokei Avraham” and continues from there with kavanah (Mishnah Berurah 101:4, in the name of the Chayei Adam). If he has already said Hashem’s Name, he concludes the berachah with kavanah. It is good to go back and think the words of Birkat Avot in his heart, for, in the Rambam’s opinion, thought is considered speech (hirhur k’dibur). However, if he already went on to say “Attah gibor,” he continues to pray and must try to have kavanah while reciting the remaining berachot, especially Birkat Modim.
. Shulchan Aruch 101:1 and Mishnah Berurah 1-3. If he is so troubled that he knows that he will not be able to concentrate even for Birkat Avot, he may not recite the Amidah. Although a situation in which a person knows ahead of time that he cannot concentrate in Birkat Avot is uncommon, the principle is that one may not pray knowing in advance that he will not have the proper kavanah in Birkat Avot. According to the Shulchan Aruch, even if he already prayed and did not have kavanah in “Avot,” he must repeat the Amidah; all the more so he may not start praying without kavanah.
.The Acharonim explain that in practice, even if a person did not have kavanah at all while reciting the Amidah, since he did intend to fulfill the mitzvah of prayer, he fulfills his obligation. So writes the Shibolei HaLeket 17, in the name of the Rishonim. Similarly, the Kaf HaChaim 101:4, in the name of Chesed L’Avraham, writes that concerning one who prays without kavanah, his berachot are not in vain. This means that what he recited is considered prayer; however, because he lacked kavanah, the Chachamim maintain that it is necessary for him to repeat the Amidah. The proof for this is that one who realizes in the middle of the Amidah that he did not have kavanah in the first berachah does not immediately go back. This is not true regarding one who realizes that he mistakenly mentioned rain in the summer, for he needs to go back immediately. Based on this, it is understandable why today we do not repeat the Amidah. See Yalkut Yosef, part 1, p. 157 and Yabia Omer, part 3, 9:3. Regarding a person who almost always has kavanah but this time did not, and he is certain that when he repeats his prayer he will have kavanah, he is permitted to go back and repeat the prayer while concentrating. It is best for him to make a stipulation that if he is exempt from repeating his prayer due to the minhag not to repeat it, his prayer should be considered a voluntary prayer (tefillat nedavah).
The Mishnah Berurah 101:4 writes, in the name of the Chayei Adam, that if he realizes before the conclusion of Birkat Avot, he must go back and start from “Elokei Avraham.” See Bei’ur Halachah s.v. “V’Ha’idna” where he suggests that someone who already finished reciting the first berachah wait for the Amidah repetition so that he can fulfill the obligation of the berachah by hearing the chazan. There are those who expressed doubt as to how he is able to fulfill his obligation of the first berachah by hearing the chazan and the rest of the Amidah by saying it himself, since the first three berachot are considered one unit (as brought in the name of the Chazon Ish). Yalkut Yosef, part 1, p. 157 writes that he continues praying and tries to have kavanah in Modim, for some say that the essence of kavanah must be either in Avot or in Modim.