Before informing your manager. you need to do a self assessment on the following.
What is that you want to become or achieve - Your long term objective.
Will that role contribute in achieving your career progression - Long term objective.
What is that you would learn and contribute to the moving team.
Will that help you in enhancing your technical / supervisory skills.
If you have a fear that your manager will not allow then you possess a specific skill which your manager has identified in you. Identify what is that skill.
When you list down your objective and road map to achieve. Have a one to one discussion with your present manager.
Ask him your career progression. How long will it take to reach your objective.
Then Present the opportunity to the manager and ask for his guidance.
You will always see green pastures on the other side. But when you go there you should not remorse that your earlier team was good. Clear your doubt before taking a decision. Once you had taken a decision there should not be any doubt.
Wishing you all the best.