Those are the ones of whom Allāh knows what is in their hearts, so turn away from them but admonish them and speak to them a far-reaching [i.e., effective] word.
Referring to Other than the Qur'an and Sunnah for Judgment is Characteristic of Non-Muslims
Allah chastises those who claim to believe in what Allah has sent down to His Messenger and to the earlier Prophets, yet they refer to other than the Book of Allah and the Sunnah of His Messenger for judgment in various disputes. It was reported that the reason behind revealing this Ayah was that a man from the Ansar and a Jew had a dispute, and the Jew said, "Let us refer to Muhammad to judge between us." However, the Muslim man said, "Let us refer to Ka`b bin Al-Ashraf (a Jew) to judge between us." It was also reported that the Ayah was revealed about some hypocrites who pretended to be Muslims, yet they sought to refer to the judgment of Jahiliyyah. Other reasons were also reported behind the revelation of the Ayah. However, the Ayah has a general meaning, as it chastises all those who refrain from referring to the Qur'an and Sunnah for judgment and prefer the judgment of whatever they chose of falsehood, which befits the description of Taghut here. This is why Allah said,
The fourth verse (63) comes up with the answer. It says that Allah is all-aware of the disbelief and hypocrisy rooted in their hearts. All their interpretations are false, and their oaths, an exercise in lying. So, the Holy Prophet ﷺ ، is being asked to ignore their excuses and reject their charge of excess against Sayyidna ` Umar , as the disbelief of this particular hypocrite had become manifest.
After that, towards the end of this verse (63), the Holy Prophet 4 has been asked to give good counsel to the rest of the hypocrites as well, in a way that may touch their hearts, that is, tell them to fear what lies beyond the present life and invite them to follow the path of Islam sincerely, or remind them of their likely worldly punishment in the event they stuck to their hypocrisy which might be exposed some day, some time, and they might as well meet the same fate as did Bishr, the hypocrite.
Tazkirul Quran
Maulana Wahiduddin KhanOn ayahs 63–64
During the early period of Madinah there used to be two courts where judgement on controversial issues might be sought: one pre-dating Islam and controlled by the leaders of the Jews, and the other set up by the Prophet, established after the Migration. Those Muslims who were not ready to sacrifice their interests for the sake of religion and were aware that their cases were too flimsy to be awarded favourable decisions by the court of the Prophet, used to approach the court of Ka‘b ibn Ashraf, a leader of the Jews. Such an attitude is totally against the spirit of the faith. If an individual is unwilling to accept God’s judgement, and aspires to be favoured with a decision to his own liking, his claim to faith is false, whatever repertoire of beautiful words he may draw on to justify his stand. However, while avoiding confrontation with such people, efforts to reform them must be effectively continued.
Uthmani text of the Madinah mushaf (King Fahd Complex). Translation: Saheeh International. Tafsir via Quran.com.