Content uploaded by Ajaya Singh
Author content
All content in this area was uploaded by Ajaya Singh on Oct 29, 2018
Content may be subject to copyright.
arXiv:1808.00875v1 [math.DS] 1 Aug 2018
MANUSCRIPT
A Study of Fatou Set, Julia set and Escaping Set
in Nearly Abelian Transcendental Semigroup
Bishnu Hari Subedi and Ajaya Singh
Abstract. We mainly generalize the notion of abelian transcendental semigroup
to nearly abelian transcendental semigroup. We prove that Fatou set, Julia set
and escaping set of nearly abelian transcendental semigroup are completely in-
variant. We investigate no wandering domain theorem in such a transcendental
semigroup. We also obtain results on a complete generalization of the classifica-
tion of periodic Fatou components.
1. Introduction
1.1. A short review of classical transcendental dynamics. Throughout
this paper, we denote the complex plane by Cand set of integers greater than zero
by N. We assume the function f:C→Cis transcendental entire function unless
otherwise stated. For any n∈N, fnalways denotes the nth iterates of f. Let f
be a transcendental entire function. The set of the form
I(f) = {z∈C:fn(z)→ ∞ as n→ ∞}
is called an escaping set and any point z∈I(S) is called escaping point. For
transcendental entire function f, the escaping set I(f) was first studied by A.
Eremenko [9]. He showed that I(f)6=∅; the boundary of this set is a Julia set J(f)
(that is, J(f) = ∂I(f)); I(f)∩J(f)6=∅; and I(f) has no bounded component.
By motivating from this last statement, he posed a question: Is every component
of I(f)unbounded?. This question is considered as an important open problem of
transcendental dynamics and nowadays is famous as Eremenko’s conjecture. Note
that the complement of Julia set J(f) in complex plane Cis a Fatou set F(f).
Recall that the set C(f) = {z∈C:f′(z) = 0}is the set of critical points
of the transcendental entire function fand the set CV (f) = {w∈C:w=
f(z) such that f′(z) = 0}of all images of all critical points is called the set of
critical values. The set AV (f) consisting of all w∈Csuch that there exists a curve
2010 Mathematics Subject Classification. 37F10, 30D05.
Key words and phrases. Transcendental semigroup, escaping set, nearly abelian semigroup.
Supported by ...
This research work of first author is supported from PhD faculty fellowship of University
Grants Commission, Nepal.
1
2 B. H. SUBEDI AND A. SINGH
(asymptotic path) Γ : [0,∞)→Cso that Γ(t)→ ∞ and f(Γ(t)) →was t→ ∞ is
called the set of asymptotic values of fand the set SV (f) = (CV (f)∪AV (f)) is
called the singular values of f. If SV (f) has only finitely many elements, then fis
said to be of finite type. If SV (f) is a bounded set, then fis said to be of bounded
type. The sets
S={f:fis of finite type}and B={f:fis of bounded type}
are respectively called Speiser class and Eremenko-Lyubich class.
1.2. Brief review of transcendental semigroup dynamics. We confine
our study on Fatou set, Julia set and escaping set of transcendental semigroup. It
is very obvious fact that a set of transcendental entire maps on Cnaturally forms
a semigroup. Here, we take a set Aof transcendental entire maps and construct a
semigroup Sconsists of all elements that can be expressed as a finite composition of
elements in A. We call such a semigroup Sby transcendental semigroup generated
by set A. A non-empty subset Tof holomorphic semigroup Sis a subsemigroup of
Sif f◦g∈Tfor all f, g ∈T. Our particular interest is to study of the dynamics
of the families of transcendental entire maps. For a collection F={fα}α∈∆of such
maps, let
S=hfαi
be a transcendental semigroup generated by them. The index set ∆ to which α
belongs is allowed to be infinite in general unless otherwise stated. Here, each f∈S
is a transcendental entire function and Sis closed under functional composition.
Thus, f∈Sis constructed through the composition of finite number of functions
fαk,(k= 1,2,3, . . . , m). That is, f=fα1◦fα2◦fα3◦ · · · ◦ fαm.
A semigroup generated by finitely many transcendental functions fi,(i= 1,2,
...,n) is called finitely generated transcendental semigroup. We write S=hf1, f2,
...,fni. If Sis generated by only one transcendental entire function f, then Sis
cyclic transcendental semigroup. We write S=hfi. In this case, each g∈Scan
be written as g=fn, where fnis the nth iterates of fwith itself. Note that in our
study of semigroup dynamics, we say S=hfiatrivial transcendental semigroup.
The transcendental semigroup Sis abelian if fi◦fj=fj◦fifor all generators fi
and fjof S. The transcendental semigroup Sis bounded type (or finite type) if each
of its generators fiis bounded type (or finite type).
The family Fof complex analytic maps forms a normal family in a domain
Dif given any composition sequence (fα) generated by the member of F, there
is a subsequence (fαk) which is uniformly convergent or divergent on all compact
subsets of D. If there is a neighborhood Uof the point z∈Csuch that Fis
normal family in U, then we say Fis normal at z. If Fis a family of members
from the transcendental semigroup S, then we simply say that Sis normal in the
neighborhood of zor Sis normal at z.
Let fbe a transcendental entire map. We say that fiteratively divergent at
z∈Cif fn(z)→ ∞ as n→ ∞. A sequence (fk)k∈Nof transcendental entire maps
is said to be iteratively divergent at zif fn
k(z)→ ∞ as n→ ∞ for all k∈N.
A STUDY OF FATOU SET, JULIA SET AND ESCAPING SET IN NEARLY ABELIAN... 3
Semigroup Sis iteratively divergent at zif fn(z)→ ∞ as n→ ∞. Otherwise, a
function f, sequence (fk)k∈Nand semigroup Sare said to be iteratively bounded at
z.
Based on the Fatou-Julia-Eremenko theory of a complex analytic function, the
Fatou set, Julia set and escaping set in the settings of transcendental semigroup are
defined as follows.
Definition 1.1 (Fatou set, Julia set and escaping set).Fatou set of the
transcendental semigroup Sis defined by
F(S) = {z∈C:Sis normal in a neighborhood of z}
and the Julia set J(S)of Sis the compliment of F(S)where as the escaping set of
Sis defined by
I(S) = {z∈C:Sis iteratively divergent at z}.
We call each point of the set I(S)by escaping point.
It is obvious that F(S) is the largest open subset Con which the family Fin
S(or semigroup Sitself) is normal. Hence its compliment J(S) is a smallest closed
set for any semigroup S. Whereas the escaping set I(S) is neither an open nor a
closed set (if it is non-empty) for any transcendental semigroup S. Any maximally
connected subset Uof the Fatou set F(S) is called a Fatou component.
If S=hfi, then F(S), J(S) and I(S) are respectively the Fatou set, Julia set
and escaping set in classical transcendental dynamics. In this situation we simply
write: F(f), J(f) and I(f).
In [16, Theorem 4.1] and [24, Theorem 2.3], it was shown that escaping set of
transcendental semigroup is in general forward invariant. However in [17, Theorem
2.1] and [24, Theorem 2.6], it was shown under certain condition that escaping
set of transcendental semigroup is backward invariant. We proved under certain
condition that Fatou set, Julia set and escaping set of transcendental semigroup
respectively equal to the Fatou set, Julia set and escaping set of its subsemigroup
[26, Theorems 1.1 and 3.1]). In [24, Theorems 3.2 and 3.3] we proved that Fatou
set, Julia set and escaping set of transcendental semigroup respectively equal to
the Fatou set, Julia set and escaping set of each of its function if semigroup Sis
abelian and in such case these sets are completely invariant. There is a slightly
larger family of transcendental semigroups that can fulfill this criteria. We call
these semigroups nearly abelian and it is considered the more general form than
that of abelian semigroups.
Definition 1.2 (Nearly abelian semigroup).We say that a transcendental
semigroup Sis nearly abelian if there is a family Φ = {φi}of conformal maps of
the form az +bfor some non-zero asuch that
(1) φi(F(S)) = F(S)for all φi∈Φand
(2) for all f, g ∈S, there is a φ∈Φsuch that f◦g=φ◦g◦f.
Note that particular example of nearly abelian semigroup is a abelian semi-
groups. Abelian semigroup follows trivially from nearly abelian semigroup if we
4 B. H. SUBEDI AND A. SINGH
choose φan identity function. The nearly abelian semigroups are the simple exam-
ples of semigroups which behave likely the same way as the classical trivial semi-
groups. In this regards, the chief aim of this paper is to prove the following result
which we have considered a strongest result of transcendental semigroup dynamics.
Theorem 1.1.Let Sbe a nearly abelian transcendental semigroup. Then for
each g∈S, we have I(S) = I(g),J(S) = J(g)and F(S) = F(g).
The chief consequence of nearly abelian transcendental semigroup is attached
with wandering domains and the concept of nearly abelian semigroups is quite useful
for the classification of periodic component (stable basin) of the Fatou set F(S).
Definition 1.3 (Stablizer, wandering component and stable domains).
For a holomorphic semigroup S, let Ube a component of the Fatou set F(S)and
Ufbe a component of Fatou set containing f(U)for some f∈S. The set of the
form
SU={f∈S:Uf=U}
is called stabilizer of Uon S. If SUis non-empty, we say that a component U
satisfying Uf=Uis called stable basin for S. The component Uof F(S)is called
wandering if the set {Uf:f∈S}contains infinitely many elements. That is, Uis
a wandering domain if there is sequence of elements {fi}of Ssuch that Ufi6=Ufj
for i6=j. Furthermore, the component Uof F(S)is called strictly wandering if
Uf=Ugimplies f=g. A stable basin Uof a holomorphic semigroup Sis
(1) attracting if it is a subdomain of attracting basin of each f∈SU
(2) supper attracting if it is a subdomain of supper attracting basin of each
f∈SU
(3) parabolic if it is a subdomain of parabolic basin of each f∈SU
(4) Siegel if it is a subdomain of Siegel disk of each f∈SU
(5) Baker if it is a subdomain of Baker domain of each f∈SU
(6) Hermann if it is a subdomain of Hermann ring of each f∈SU
In classical holomorphic iteration theory, the stable basin is one of the above
types but in transcendental iteration theory, the stable basin is not a Hermann be-
cause a transcendental entire function does not have Hermann ring [13, Proposition
4.2].
Note that for any rational function f, we always have Uf=U. So USis non-
empty for a rational semigroup S. However, if fis transcendental, it is possible that
Uf6=U. So, USmay be empty for transcendental semigroup S. Bergweiler and
Rohde [8] proved that Uf−Ucontains at most one point which is an asymptotic
value of fif fis an entire function.
We prove the following no wandering domain result which is analogous to the
rational semigroups [12, Theorem 5.1].
Theorem 1.2.Let Sbe a nearly abelian semigroup generated by transcendental
entire functions of finite or bounded type. Then F(S)has no wandering domain.
A STUDY OF FATOU SET, JULIA SET AND ESCAPING SET IN NEARLY ABELIAN... 5
We prove the following result regarding the classification of periodic components
of Fatou set F(S).
Theorem 1.3.Let Ube a component of the Fatou set F(S)of the nearly abelian
semigroup Sgenerated by transcendental entire functions of finite or bounded type
and Vbe a subset F(S)containing in the forward orbit of U. Then Vis attracting,
super attracting, parabolic, Siegel or Baker.
The organization of this paper is as follows: In section 2, we briefly review no-
tion of nearly abelian transcendental semigroup with suitable examples. We prove
existence theorem for nearly abelian transcendental semigroup. We also investi-
gate necessary condition of any transcendental semigroup to be nearly abelian. In
section 3, we mainly prove theorem 1.1. and theorem 1.2. In section 4, we prove
classification theorem (Theorem1.3) of periodic Fatou component of transcendental
semigroup.
2. The Notion of Nearly Abelian Transcendental Semigroup
In this section, we extend the results of abelian transcendental semigroups to
more general settings of nearly abelian transcendental semigroups. The principal
feature of nearly abelian rational semigroup was investigated by Hinkannen and
Martin [12, Theorem 4.1]. In such a case, they found that the Julia set J(S)
of rational semigroup Sis same as Julia set J(f) of each f∈S. Indeed, this is
generalization of the result of Fatou [11] and Julia [15] (if rational maps fand gare
permutable, then they have the same Julia sets) in semigroup settings. However, the
corresponding result may not hold in the case of permutable transcendental entire
functions but in nearly abelian settings of transcendental semigroup, we found that
the corresponding result of Hinkannen and Martin [12, Theorem 4.1] holds.
The definition 1.2 of nearly abelian transcendental semigroup looks more re-
strictive on the affine map of the form φ(z) = az +b, a 6= 0 and this type of
function can play the role of semiconjugacy to certain class of transcendental entire
functions. Recall that function fis (semi) conjugate to the function gif there is a
continuous function φsuch that φ◦f=g◦φ. For example, transcendental entire
function f1(z) = λcos zis semi-conjugate to another transcendental entire function
f2(z) = −λcos zbecause there is a function φ(z) = −zsuch that φ◦f1=f2◦φ.
If there is a transcendental semigroup generated by such type of semi-conjugate
functions, then semigroup will more likely to be nearly abelian.
Theorem 2.1.Let S=hf1, f2,...fn,...ibe a transcendental semigroup and let
φbe an entire function of the form z→az +bfor some non zero awith a, b ∈C
such that φ◦fi=fj◦φfor all fiand fjwith i6=j. If φ◦f=gfor all f∈S.
Then the transcendental semigroup Sis nearly abelian.
To prove this theorem 2.1, we need the following lemmas.
Lemma 2.1.Let S=hf1, f2,...fn,...ibe a transcendental semigroup and let φ
be an entire function of the form z→az +bfor some non zero awith a, b ∈C. If
φ◦fi=fj◦φfor all fiand fjwith i6=j, then φ(F(S)) = F(S)and φ(J(S)) = J(S).
6 B. H. SUBEDI AND A. SINGH
Proof. First of all, we prove that if φ◦fi=fj◦φfor all iand jwith i6=j,
then φ◦f=g◦φfor all f, g ∈S.
Since any f, g ∈Scan be written as f=fi1◦fi2◦...◦finand g=fj1◦fj2◦...◦fjn.
Now φ◦f=φ◦fi1◦fi2◦...◦fin=fj1◦φ◦◦fi2◦...◦fin=...=fj1◦fj2◦...◦fjn◦φ=
g◦φ. This proves our claim.
Let w∈φ(F(S)). Then there is z0∈F(S) such that w=φ(z0). Let U⊂F(S)
is a neighborhood of z0such that |f(z)−f(z0)|< ǫ/2 for all z∈Uand f∈S.
This shows that f(U) has diameter less than ǫfor all f∈S. Since function φ
has bounded first derivative a6= 0, so it is a Lipschitz with Lipschitz constant
k= sup |φ′(z)|=a. Now for any g∈S, the diameter of g(φ(U)) = φ(f(U)) is less
than kǫ. Hence w=φ(z0)∈F(S). This shows that φ(F(S)) ⊂F(S).
Next, let w∈φ(J(S)). Then w=φ(z0) for some z0∈J(S). Let z0be a
repelling fixed point for some f∈Sbut which is not a critical point of φ, then
φ◦f=g◦φgives ghas a fixed point at φ(z0) with same multiplier as that of fat z0.
Thus φmaps repelling fixed points of any f∈Sto repelling fixed points of another
g∈S. Since from [20, Theorem 4.1 and 4.2], Julia set of transcendental semigroup
is perfect and J(S) = ∪f∈SJ(f)), where repelling periodic points are dense in J(f)
for each f∈S. So by above discussion, it then follows that φ(J(S)) ⊂J(S).
Finally, since φ(C) = C. Using this fact in F(S) = C−J(S) and J(S) =
C−F(S), we get φ(F(S)) = C−φ(J(S)) and φ(J(S)) = C−φ(F(S)). Again
using facts φ(J(S)) ⊂J(S) and φ(F(S)) ⊂F(S) in φ(F(S)) = C−φ(J(S))
and φ(J(S)) = C−φ(F(S)) respectively, we will get required opposite inclusions
F(S)⊂φ(F(S)) and J(S)⊂φ(J(S)).
Note that this lemma 2.1 tells us that the first condition φi(F(S)) = F(S) of
nearly abelian semigroup can be replaced by (semi) conjugacy relation φ◦fi=fj◦φ
for all fiand fjwith i6=j. This is a way that one can replace the first condition
of the definition.
Proof of the Theorem 2.1. The first part for nearly abelian semigroup fol-
lows from the lemma 2.1.
The second part follows from the following simple calculations. The hypothesis
φ◦fi=fj◦φfor all fiand fjwith i6=jgives f◦φ=φ◦gfor all f, g ∈Sand
from the hypothesis φ◦f=gfor all f∈S, we can write φ◦g◦f=f◦φ◦f=f◦g
for all f, g ∈S.
There are general and particular examples of transcendental entire functions
that fulfills the essence of above theorem 2.1 and the semigroup generated by these
functions is nearly abelian.
Example 2.1.Let φbe an entire function of the form z→ −z+cfor some
c∈C. Let fbe a transcendental entire function with f◦φ=fand function gis
defined by g=φ◦f. Then functions fand gare conjugates and the semigroup
S=hf, gigenerated by these two functions fand gis nearly abelian.
A STUDY OF FATOU SET, JULIA SET AND ESCAPING SET IN NEARLY ABELIAN... 7
Solution. Let f, g and φbe as in the statement of the question. It is clear that
φ2= Identity. Then g◦φ=φ◦f◦φ=φ◦f. This proves that functions fand gare
conjugates. The condition φ(F(S)) = F(S) for all φ∈Φ of the definition of nearly
abelian semigroup follows from above lemma 2.1. The second condition follows
from the theorem 2.1. More explicitly it follows from the following calculation.
f◦g=f◦φ◦f=f◦f=f2=φ2◦f2=φ◦φ◦f◦f=φ◦g◦f.
Therefore, the semigroup S=hf, gigenerated by these two functions fand gis
nearly abelian. From the fact g=φ◦f, we can say that φis not an identity.
Example 2.2.Let f(z) = ez2+λ, and g=φ◦fwhere φ(z) = −z. Then the
semi group S=hf, giis nearly abelian. Like wise, functions f(z) = λcos zand
g=φ◦fwhere φ(z) = −zgenerate the nearly abelian semigroup.
Solution. The given functions in the question fulfills all conditions such as
f◦φ=f,φ2= identity as well as φ◦f=g◦φof above theorem 2.1 as well as
example 2.1. Therefore, the semigroup S=hf, giis indeed nearly abelian. Note
that φ◦f=−f6=f, so φis not an identity.
Note that the above example 2.1 is just for a nice general example of above the-
orem 2.1 that says there is an nearly abelian transcendental semigroup. Unfortu-
nately, this example is not generating many more examples of transcendental entire
functions that can generate transcendental semigroup. Basically, it generates even
functions or translation of even functions. For example: If we set h(z) = f(z+c
2),
then h(z) = f(z+c
2) = (f◦φ)(z) = f(c−z−c
2) = f(c
2−z) = h(−z). That is, h
is an even function.
Above theorem 2.1 provided a criterion to be a nearly abelian transcendental
semigroup. Are there other criteria that help us to make nearly abelian tran-
scendental semigroups and somehow connected to above criterion? In the case of
rational semigroup S, there is very strong criteria for nearly abelian semigroup due
to Hinkkanen and Martin [12, Corollary 4.1]. This criteria is possible because of
Beardon’s result [5, Theorem 1]. It states the following. For any two polynomials
fand gand degree of fis at least two, then J(f) = J(g)if and only if there is a
map φ(z) = az +bwith |a|= 1 such that f◦g=φ◦g◦f. Our aim in this context
to see a transcendental semigroup Sthat fulfills this results. Similar to Hinkkanen
and Martin [12, Corollary 4.1], we have formulated the following theorem.
Theorem 2.2.Let Sbe a transcendental semigroup and suppose that I(f) =
I(g)for all f, g ∈S. Then Sis nearly abelian semigroup.
According to this theorem, the condition I(f) = I(g) for all f, g ∈Sis very
strong one that replace both of conditions of the definition 1.2 of nearly abelian
semigroup. Indeed, the condition I(f) = I(g) stated in this theorem 2.2 can be used
to obtain the converse of Baker’s question [3], namely, for two distinct permutable
transcendental entire functions fand g, does it follow J(f) = J(g)? This question is
itself a difficult one of classical transcendental dynamics to answer. The converse of
8 B. H. SUBEDI AND A. SINGH
this Baker’s question is again difficult to settle down. We expect that this will settle
down in nearly abelian transcendental semigroups. We completely characterize first
all permutable transcendental entire functions. For the given transcendental entire
function flet us define the following three classes of algebraic structures:
S(f) = {g:J(g) = J(f)},
C(f) = {g:f◦g=g◦f},
Σ(f) = {φ∈Φ : φ(J(f)) = J(f)},
where Φ is a group of conformal isometrics φ(z) = az +bwith |a|= 1. As Julia set
J(f) of any transcendental function fis unbounded, each element of Φ is translation
or rotation or both. Note that if both fand gwere polynomials, Beardon [5,
Theorem 1] proved that S(f) = {g:f◦g=φ◦g◦ffor some φ∈Σ(f)}. In terms of
the result of Fatou [11], Julia [15], Baker and Eremenko [4, Theorem 1], Beardon [5,
Theorem 2] also proved that each of sets S(f) and C(f) is a semigroup, C(f)⊂ S(f)
and C(f) = S(f) when Σ(f) is trivial. Unfortunately, analogous result may not
hold if fand gare transcendental entire functions. So we need further complete
classification of all pair of permutable transcendental entire functions that have
same Julia sets. Only known characterization of transcendental entire functions f
and gthat can have same Julia set are as follows:
(1) if function g=fnfor some n(>2) ∈N;
(2) if fand gare permutable functions such that g(z) = af(z) + b, where
a(6= 0,|a|= 1) and bare complex constants;
(3) if fand gare permutable functions and p(z) be a non-constant polynomial
such that p(g(z)) = ap(f(z))+b, where a(6= 0) and bare complex constants;
(4) if fand gare permutable functions of bounded type;
(5) if fand gare permutable functions without wandering domains:
(6) if g(z) = af n(z) + b, where |a|= 1 and b∈C;
(7) if fm(z) = gn(z) for some m, n ∈N.
The functions fand gstated above belong to class C(f) with same Julia sets and
in such functions, we can write C(f)⊂ S(f). However, in general C(f)∩ S(f)6=∅.
Baker[1] and Iyer [14] investigated that if non-constant polynomial fpermutes
with transcendental entire function g, then f(z) = e2mπi/kz+bfor some m, k ∈N
and complex number b. That is, commuting polynomials of any transcendental
entire function fare from the group Σ(f) of symmetries of J(f). On the other
hand, some transcendental entire function can have a linear factor. For example,
ez+z, eez+z, zezare transcendental entire functions have a linear factor. More
generally, eaz+b+p(z), where p(z) is a non-constant polynomial, a(6= 0) and bare
two complex constants has a linear factor. Note that such type functions are known
as prime functions in the entire sense. More generally, an entire function fis prime
(left prime) in the entire sense if f(z) = g(h(z)) for some entire functions gand h,
then either gor his linear (gis linear whenever his transcendental). Note that
if q(z) is periodic entire function of finite lower order and p(z) is a non-constant
polynomial, then q(z)+p(z) is prime in the entire sense. ez+p(z) and sin z+p(z) are
A STUDY OF FATOU SET, JULIA SET AND ESCAPING SET IN NEARLY ABELIAN... 9
examples of prime functions. These functions are nice examples of of transcendental
entire functions belongs to category (6) stated above and so any entire function g
that commutes with such functions can have same Julia sets. More detail study of
above stated class of transcendental entire functions as well as other related results
can be found in [19, Theorems 1, 2 and 3], [21, Lemma 2.1, 2.2 and Theorems 2.1]
[22, Theorems 1, 2, 3 and 4] and [27, Theorems 1, 2 and 3]. Further, more recent
analysis have been made by Benini, Rippon and Stallard [7, Theorems 1.1, 1.2 and
1.3].
The chief concern of this paper has to consider the converse question: When
do two transcendental entire functions have the same Julia set? That is, for two
transcendental entire functions fand g, if J(f) = J(g), then what will be the
proper relation between fand g? To prove theorem 2.2, we need the following
lemma which is analogous to [12, Theorem 4.2] of rational semigroup. It proves
there is a relation of virtually abelian between the transcendental entire functions
that have the same Escaping set. Note that Julia set is the boundary of escaping
set, so whatever result hold for escaping set, the same type of result hold for Julia
set.
Lemma 2.2.let fand gbe transcendental entire functions. If I(f) = I(g), then
there is a map φ(z) = az +bwith |a|= 1 such that f◦g=φ◦g◦f.
Proof. We prove this lemma on the basis of the sequence of the following facts.
(1) Functions fand gas stated in the lemma, the following statements hold:
(a) I(f) is completely invariant under g.
(b) J(f) is completely invariant under g.
(c) J(f) = J(g).
(d) S(f) is a semigroup.
From I(f) = I(g), (a) follows easily. (b) follows as ∂I(f) = J(f)
(boundary of the completely invariant set under the same function is
completely invariant (see for instance [6, Theorem 3.2.3])). (c) follows
from the given I(f) = I(g). For (d), let us suppose g1, g2∈ S(f). By
(b) J(f) is completely invariant under both g1and g2and hence it is
completely invariant under g1◦g2. By (c) J(f) = J(g1◦g2). This
proves g1◦g2∈ S(f) and hence S(f) is a semigroup.
(2) For φ∈Φ and functions fand gas above, the following facts hold:
(a) φ◦g∈ S(f) and g◦φ∈ S(f).
(b) f◦g∈ S(f), g◦f∈ S(f) and φ◦g◦f∈ S(f).
From 1(b), J(f) is completely invariant under gand φis a symmetry
of J(f), so J(f) is completely invariant under both φ◦gand g◦φ.
By 1(c), J(f) = J(φ◦g) and J(f) = J(g◦φ). This follows (a). Since
J(f) is completely invariant under fand by 1(b), it is also completely
invariant under g, and so it is completely invariant under f◦gand
g◦f. As φis a symmetry of J(f), J(f) is completely invariant under
φ◦g◦f. By 1(c), J(f◦g) = J(φ◦g◦f). This proves (b).
(3) f,gand φas above, then
10 B. H. SUBEDI AND A. SINGH
(a) S(f) = {g:f◦g=φ◦g◦ffor some φ∈Σ(f)}.
(b) S(f) = {g:f◦g=g◦f}.
Since fand gare transcendental entire functions, so from 2(b), J(f◦
g) = J(φ◦g◦f) must imply f◦g=φ◦g◦f. This follows (a). If φ
is a trivial symmetry, (b) follows.
Proof of the Theorem 2.2. Since φis a symmetry of Julia set J(f), so
φ(J(f)) = J(f). If we apply φon F(f) = C−J(f), we get φ(F(f)) = F(f). The
second part of the nearly abelian semigroup follows from lemma 2.2.
3. Proof of the Theorems 1.1 and 1.2
Hinkkanen and Martin [12, Theorem 4.1] proved that the Julia set of the nearly
abelian rational semigroup is same as Julia set of each of its function. Indeed, this
is a generalization of the result of abelian rational semigroup that we prove in [24,
Theorem 3.1]. It will be difficult to say the same in general if we take abelian
transcendental semigroup. That is, if we have abelian transcendental semigroup S,
it would not always J(S) = J(f) for all f∈S. It would be sometime in certain
case, and one of the case was proved by K.K. Poon [20, Theorem 5.1].
Theorem 3.1.Let S=hf1, f2,...fniis an abelian finite type transcendental
semigroup. Then F(S) = F(f)for all f∈S.
Indeed this result looks like extension work of the following results of A. P. Singh
and Yuefei Wang [22, Theorems 2, 3] of classical transcendental dynamics.
Theorem 3.2.Let fand gare two permutable transcendental entire maps. If
both fand ghave no wandering domains, then J(f) = J(f◦g) = J(g).
Theorem 3.3.Let fand gare two permutable transcendental entire maps. If
both fand gare of bounded type, then J(f) = J(f◦g) = J(g).
Our particular interest is how far the result of K.K. Poon [20, Theorem 5.1] can
be generalized to nearly abelian transcendental semigroup. We prove the following
more general result in the case of nearly abelian transcendental semigroup S. It is
indeed, the converse of the theorem 2.2.
Lemma 3.1.Let Sbe a transcendental semigroup. Then
(1) int(I(S)) ⊂F(S)and ext(I(S)) ⊂F(S), where int and ext respectively
denote the interior and exterior of I(S).
(2) ∂I (S) = J(S), where ∂I(S)denotes the boundary of I(S).
Proof. We refer for instance lemma 4.2 and theorem 4.3 of [16].
Note that this lemma 3.1 is a extension of Eremenko’s result [9], ∂I(f) = J(f)
of classical transcendental dynamics to more general semigroup settings.
A STUDY OF FATOU SET, JULIA SET AND ESCAPING SET IN NEARLY ABELIAN... 11
Proof of the Theorem 1.1. We prove I(S) = I(f) for all f∈Sand the
reaming results follows from lemma 3.1.
By [25, Theorem 1.3], I(S)⊆I(g) for any g∈S. For opposite inclusion,
suppose z∈I(g), then by the definition of escaping set, gn(z)→ ∞ as n→ ∞.
Since semigroup Sis nearly abelian so for all f, g ∈Sthere is φ∈Φ such that
f◦g=φ◦g◦f. By induction, we easily get f◦gn= (φ◦g)n◦f. Thus for any
z∈I(g), we have (f◦gn)(z) = ((φ◦g)n◦f)(z) = (hn◦f)(z) = (hn(f(z)), where
φ◦g=h∈S. As gn(z)→ ∞, then hn(z)→ ∞ for all h∈S. So, hn(f(z)) → ∞
as n→ ∞. This implies that f(z)∈I(h) = I(φ◦g)⊂I(φ)∪I(g). This gives
either f(I(g)) ⊂I(φ) or f(I(g)) ⊂I(g)) for all f, g ∈S. Therefore, z∈I(S). Thus
I(g)⊆I(S). Hence, I(S) = I(g) for all g∈S.
It is known that wandering domains do not exist in the case of rational functions
and polynomials but transcendental entire functions may have wandering domains.
However, in generalized settings of semigroups, rational (or polynomial) semigroups
may have wandering domains ([12, Theorem 5.2]). In this section, we investigate
that there are transcendental semigroups that may have wandering domains. This
investigation is possible via nearly abelian transcendental semigroups. In particular,
the theorem 1.1 is useful to prove no wandering domain theorem of transcendental
semigroups.
Proof of the Theorem 1.2. Since Sis a nearly abelian semigroup of tran-
scendental entire functions, so by the theorem 1.1, we have F(S) = F(f) for any
f∈S. Since each f∈Sis a transcendental entire functions of finite or bounded
type, so by the theorems 4.32 and 4.33 of [13], the Fatou set F(f) has no wandering
domain. So, F(S) has no wandering domain.
4. Classification of Stable Basins of Fatou Components
Recall that a subsemigroup Tof a holomorphic semigroup Sis said to be of
cofinite index if there exists collection of exactly n - elements {f1, f2,...,fn}of
S1=S∪ {Identity}such that for any f∈S, there is i∈ {1,2,...,n}such that
(4.1) fi◦f∈T
The smallest nthat satisfies 4.1 is called cofinite index of Tin S.
By the theorem 1.2. no wandering domains in the statement of Theorem 2.3 of
[26] can be replaced by nearly abelian semigroup as in the following result.
Theorem 4.1.Let Sbe a nearly abelian transcendental semigroup generated by
finite or bounded type transcendental entire functions. Let Ube any component of
Fatou set. Then the forward orbit {Uf:f∈S}of Uunder Scontains a stabilizer
of Uof cofinite index.
The concept of nearly abelian semigroups is quite useful for the classification of
periodic component (stable basin) of the Fatou set F(S). Recall that a stable basin
Uof a transcendental semigroup Sis
(1) attracting if it is a subdomain of attracting basin of each f∈SU
12 B. H. SUBEDI AND A. SINGH
(2) supper attracting if it is a subdomain of supper attracting basin of each
f∈SU
(3) parabolic if it is a subdomain of parabolic basin of each f∈SU
(4) Siegel if it is a subdomain of Siegel disk of each f∈SU
(5) Baker if it is a subdomain of Baker domain of each f∈SU
In classical transcendental iteration theory, the stable basin is one of the above
types but not Hermann because a transcendental entire function does not have
Hermann ring [13, Proposition 4.2].
We have proved the following analogous statement of [12, Theorem 6.2] in the
case of transcendental semigroup.
Theorem 4.2.Let Ube a component of the Fatou set F(S)of the nearly abelian
semigroup Sof transcendental entire functions of finite or bounded type and Vbe
a subset F(S)containing in the forward orbit of U. Then Vis attracting, super
attracting, parabolic, Siegel or Baker.
Proof. By the theorem 4.1, V⊂F(S) is a stable basin of cofinite index, so,
Vf=Vfor some f∈S. By the theorem 1.1, F(S) = F(f) for any f∈S. By the
theorem 4.5 (1, 2, 3, 5) of [13], Vis a attracting (or supper attracting), parabolic,
Siegel or Baker domain of F(f) = F(S).
Remark 4.1.(1) In above theorem, Baker domain does not exist if Stran-
scendental semigroup of bounded type. The fact is obvious. That is,
for transcendental nearly abelian semigroup Sof bounded type, We have
F(S) = F(f) for any f∈S. However, for bounded type transcenden-
tal entire function f, Fatou set F(f) does not contain Baker domain [13,
Theorem 4.29].
(2) Under the same condition as above, all components of F(S) are simply
connected. The fact is obvious. As above, we have F(S) = F(f) for
any f∈S. From [10, Proposition 3], all components of F(f) are simply
connected if f∈B.
In the classical transcendental dynamics, if stable domains of a transcendental
entire function fare bounded, then the Fatou Set F(f) does not contain asymptotic
values. This fact holds good in transcendental semigroup if it is nearly abelian.
Theorem 4.3.If all stable domains of a nearly abelian transcendental semi-
group Sare bounded, then Fatou set F(S)does not contain any asymptotic values.
Proof. By the theorem 1.1, F(S) = F(f) for any f∈S. By the theorem
4.16 of [13], F(f) does not contain any asymptotic value of f. Hence the result
follows.
References
[1] Baker, I. N.: Zusammensetzungen ganzer Funktionen, Math. Z. 69 (1958), 121-163.
[2] Baker, I. N.: Permutable entire functions, Math. Z. 79 (1968), 243–249.
[3] Baker, I.N.: Wandering domains in the iterations entire functions, J. Proc. London Math.
Soc. (3) 49 (1984), no. 3, 563-576.
A STUDY OF FATOU SET, JULIA SET AND ESCAPING SET IN NEARLY ABELIAN... 13
[4] Baker, I. N. and Eremenko, A.: A problem on Julia set, Ann. Acad. Sci. Fen. , Series A. I.
Mathematica, Vol. 12, (1987), 229-236.
[5] Beardon, A. F.: Symmetries of Julia sets, Bull. London Math. Soc. 22 (1990), 576-582.
[6] Beardon, A. F.: Iteration of rational functions, Complex anlytic dynamical systems, Springer-
Verlag, New York Inc., 1991.
[7] Benini, A. M., Rippon, P. J. and Stallard, G. M. Permutable entire functions and multiply
connected wandering domains, arXiv: 1503. 08112v1 [math. DS], 27 March 2015.
[8] Bergweiler, W. and Rohde, S.: Omitted value in the domains of normality, Proc. Amer. Math.
Soc. 123 (1995), 1857-1858.
[9] Eremenko, A.: On the iterations of entire functions, Dynamical System and Ergodic Theory,
Banach Center Publication Volume 23, Warsaw, Poland, (1989).
[10] Eremenko, A., and Lyubich, M.Y.: Dynamical Properties of Some Classes of Entire Func-
tions,Ann. Inst. Fourier, Grenoble, 42 (1992), 989-1020.
[11] Fatou, P.: Sur les equations fonctionelles, Acta Math., 47 (4) (1926), 337-340.
[12] Hinkkanen, A. and Martin, G.J.: The dynamics of semi-groups of rational functions- I, Proc.
London Math. Soc. (3) 73, 358-384, (1996).
[13] Hua, X.H. and Yang, C.C.: Dynamic of transcendental functions, Gordon and Breach Science
Publication, (1998).
[14] Iyer, V. G.: On permutable integral integral functions, J. London Math. Soc. 34 (1959),
141-144.
[15] Julia, G.: Sur la permutabilite des fonctions rationnnelles, Ann. Sci. Ecole Norm. Sup 39
(1922), 131-215.
[16] Kumar, D. and Kumar, S.: The dynamics of semigroups of transcendental entire functions-II,
arXiv: 1401.0425 v3 (math.DS), May 22, 2014.
[17] Kumar, D. and Kumar, S.: Escaping set and Julia set of transcendental semigroups,
arXiv:1410.2747 v3 (math. DS) October 10, 2014.
[18] Kumar, D.: On escaping set of some families of entire functions and dynamics of composite
entire functions, arXiv: 1406.2453v4 [math. DS] 4 Oct. 2015.
[19] Ng, T. W.: Permutable entire functions and their Julia sets, Math. Proc. Camb. Phil. Soc.
131 (2001), 129-138.
[20] Poon, K.K.: Fatou- Julia theory on transcendental semi-groups, Bull. Austral. Math. Soc.
Vol- 58(1998) PP 403-410.
[21] Poon, K. K. and Yang, C. C.: Dynamical behavior of two permutable entire functions, Annales
Polonici Mathematici LXVIII 2 (1998), 159-163.
[22] Singh, A. P. and Wang, Y.: Julia sets of permutable holomorphic maps, Science in China
Series A: Mathematics, Vol. 49, No. 11 (2006), 1715-1721.
[23] Subedi, B. H. and Singh, A.: A short comparison of classical complex dynamics and holo-
morphic semigroup dynamics, arXiv: 1803.08368v1[math. DS] 22 March 2018.
[24] Subedi, B. H. and Singh, A.: Invariant properties of Fatou set, Julia set and escaping set of
holomorphic semigroup, arXiv: 1803.09662v1[math. DS] 23 March 2018.
[25] Subedi, B. H. and Singh, A.: Escaping set of hyperbolic semigroup, arXiv:1803.10381v1 [math
DS], 28 March, 2018.
[26] Subedi, B. H. and Singh, A.: Fatou set, Julia set and escaping set in holomorphic subsemi-
group dynamics, arXiv:1807.04499v1 [math. DS], 12 July, 2018.
[27] Wang, X. L., Hua, X. H., Yang, C.C. and Yang, D.: Dynamics of permutable transcendental
entire functions, Rocky Mountain J. Math. Vol. 36, No. 6, (2006), 2041-2055.
14 B. H. SUBEDI AND A. SINGH
Central Department of Mathematics, Institute of Science and Technology,
Tribhuvan University, Kirtipur, Kathmandu, Nepal
E-mail address:subedi.abs@gmail.com / subedi bh@cdmathtu.edu.np
Central Department of Mathematics, Institute of Science and Technology,
Tribhuvan University, Kirtipur, Kathmandu, Nepal
E-mail address:singh.ajaya1@gmail.com / singh a@cdmathtu.edu.np
