Content uploaded by Ajaya Singh
Author content
All content in this area was uploaded by Ajaya Singh on Oct 29, 2018
Content may be subject to copyright.
arXiv:1807.04499v1 [math.DS] 12 Jul 2018
MANUSCRIPT
Fatou Set, Julia Set and Escaping Set in
Holomorphic Subsemigroup Dynamics
Bishnu Hari Subedi and Ajaya Singh
Abstract. We investigate to what extent Fatou set, Julia set and escaping set
of transcendental semigroup is respectively equal to the Fatou set, Julia set and
escaping set of its subsemigroup. We define partial fundamental set and funda-
mental set of transcendental semigroup and on the basis of this set, we prove that
Fatou set and escaping set of transcendental semigroup Sare non-empty.
1. Introduction
We confine our study on Fatou set, Julia set and escaping set of holomorphic
semigroup and its subsemigroup defined in complex plane Cor extended complex
plane C∞. Semigroup Sis a very classical algebraic structure with a binary com-
position that satisfies associative law. It naturally arose from the general mapping
of a set into itself. So a set of holomorphic maps on Cor C∞naturally forms a
semigroup. Here, we take a set Aof holomorphic maps and construct a semigroup
Sconsists of all elements that can be expressed as a finite composition of elements
in A. We call such a semigroup Sby holomorphic semigroup generated by set A. A
non-empty subset Tof holomorphic semigroup Sis a subsemigroup of Sif f◦g∈T
for all f, g ∈T.
For our simplicity, we denote the class of all rational maps on C∞by Rand
class of all transcendental entire maps on Cby E. Our particular interest is to
study of the dynamics of the families of above two classes of holomorphic maps.
For a collection F={fα}α∈∆of such maps, let
S=hfαi
be a holomorphic semigroup generated by them. Here Fis either a collection R
of rational maps or a collection Eof transcendental entire maps (there are several
holomorphic semigroups generated by general meromorphic functions, but in this
paper, we are only interested in holomorphic semigroups generated by either rational
2010 Mathematics Subject Classification. 37F10, 30D05.
Key words and phrases. Transcendental semigroup, escaping set, finite index and co-index,
fundamental set etc.
This research work of first author is supported from PhD faculty fellowship of University
Grants Commission, Nepal.
1
2 B. H. SUBEDI AND A. SINGH
functions or transcendental entire functions). The index set ∆ to which αbelongs
is allowed to be infinite in general unless otherwise stated. Here, each f∈Sis a
holomorphic function and Sis closed under functional composition. Thus, f∈S
is constructed through the composition of finite number of functions fαk,(k=
1,2,3,...,m). That is, f=fα1◦fα2◦fα3◦ · · · ◦ fαm. In particular, if fα∈R, we
say S=hfαiarational semigroup and if fα∈E, we say S=hfαiatranscendental
semigroup.
A semigroup generated by finitely many holomorphic functions fi,(i= 1,2,...,
n) is called finitely generated holomorphic semigroup. We write S=hf1, f2,...,fni.
If Sis generated by only one holomorphic function f, then Sis cyclic semigroup.
We write S=hfi. In this case, each g∈Scan be written as g=fn, where fnis
the nth iterates of fwith itself. Note that in our study of semigroup dynamics, we
say S=hfiatrivial semigroup.
Next, we define and discuss some special collection and sequences of holomorphic
functions. Note that all notions of convergence that we deal in this paper will be
with respect to the Euclidean metric on the complex plane Cor spherical metric
on the Riemann sphere C∞.
The family Fof complex analytic maps forms a normal family in a domain
Dif given any composition sequence (fα) generated by the member of F, there
is a subsequence (fαk) which is uniformly convergent or divergent on all compact
subsets of D. If there is a neighborhood Uof the point z∈Csuch that Fis normal
family in U, then we say Fis normal at z. If Fis a family of members from the
semigroup S, then we simply say that Sis normal in the neighborhood of zor Sis
normal at z.
Let fbe a holomorphic map. We say that fiteratively divergent at z∈Cif
fn(z)→αas n→ ∞, where αis an essential singularity of f. A sequence (fk)k∈Nof
holomorphic maps is said to be iteratively divergent at zif fn
k(z)→αkas n→ ∞
for all k∈N, where αkis an essential singularity of fkfor each k. Semigroup S
is iteratively divergent at zif fn(z)→αfas n→ ∞, where αfis an essential
singularity of each f∈S. Otherwise, a function f, sequence (fk)k∈Nand semigroup
Sare said to be iteratively bounded at z. The following result will be clear from the
definition of holomorphic semigroup. It shows that every element of holomorphic
semigroup can be written as finite composition of the sequence of fα
Proposition 1.1.Let S=hfαibe an arbitrary holomorphic semigroup. Then
for every f∈S,fm(for all m∈N) can be written as fm=fα1◦fα2◦fα3◦ · · · ◦ fαp
for some p∈N.
In classical complex dynamics, each of Fatou set, Julia set and escaping set
are defined in two different but equivalent ways. In first definition, Fatou set is
defined as the set of normality of the iterates of given function, Julia set is defined
as the complement of the Fatou set and escaping set is defined as the set of points
that goes to essential singularity under the iterates of given function. The second
definition of Fatou set is given as a largest completely invariant open set and Julia
set is given as a smallest completely invariant close set whereas escaping set is a
FATOU SET, JULIA SET AND ESCAPING SET IN HOLOMORPHIC SUB... 3
completely invariant non-empty neither open nor close set in C. Each of these
definitions can be naturally extended to the settings of holomorphic semigroup S
but extension definitions are not equivalent. Based on above first definition (that
is, on the Fatou-Julia-Eremenko theory of a complex analytic function), the Fatou
set, Julia set and escaping set in the settings of holomorphic semigroup are defined
as follows.
Definition 1.1 (Fatou set, Julia set and escaping set).Fatou set of the
holomorphic semigroup Sis defined by
F(S) = {z∈C:Sis normal in a neighborhood of z}
and the Julia set J(S)of Sis the compliment of F(S). If Sis a transcendental
semigroup, the escaping set of Sis defined by
I(S) = {z∈C:Sis iteratively divergent at z}
We call each point of the set I(S)by escaping point.
It is obvious that F(S) is the largest open subset (of Cor C∞) on which the
family Fin S(or semigroup Sitself) is normal. Hence its compliment J(S) is a
smallest closed set for any semigroup S. Whereas the escaping set I(S) is neither
an open nor a closed set (if it is non-empty) for any semigroup S. Any maximally
connected subset Uof the Fatou set F(S) is called a Fatou component.
If S=hfi, then F(S), J(S) and I(S) are respectively the Fatou set, Julia set
and escaping set in classical complex dynamics. In this situation we simply write:
F(f), J(f) and I(f).
There is possibility of being Fatou set, Julia set and escaping set of holomor-
phic semigroup respectively equal to the Fatou set, Julia set and escaping set of
its subsemigroup. To get this results, we need the notion of different indexes of
subsemigroup of a semigroup S.
Definition 1.2 (Finite index and cofinite index).A subsemigroup Tof a
holomorphic semigroup Sis said to be of finite index if there exists finite collection
of elements {f1, f2,...,fn}of S1where S1=S∪ {Identity}such that
(1.1) S=f1◦T∪f2◦T∪...∪fn◦T
The smallest nthat satisfies 1.1 is called index of Tin S. Similarly a subsemigroup
Tof a holomorphic semigroup Sis said to be of cofinite index if there exists finite
collection of elements {f1, f2,...,fn}of S1such that for any f∈S, there is i∈
{1,2,...,n}such that
(1.2) fi◦f∈T
The smallest nthat satisfies 1.2 is called cofinite index of Tin S.
Note that the size of subsemigroup Tinside semigroup Sis measured in terms
of index. If subsemigroup Thas finite index or cofinite index in semigroup S, then
we say Tis finite subsemigroup or cofinite subsemigroup respectively.
4 B. H. SUBEDI AND A. SINGH
In [11, Theorems 5.1], K.K. Poon proved that Fatou set and Julia set of finitely
generated abelian transcendental semigroup Sis same as the Fatou set and Julia
set of each of its particular function if semigroup Sis generated by finite type
transcendental entire maps. In [16, Theorems 3.3], we proved that escaping set of
transcendental semigroup Sis same as escaping set of each of its particular function
if semigroup Sgererated by finite type transcendental entire maps. In this paper,
we prove the following result.
Theorem 1.1.If a subsemigroup Thas finite index or cofinite index in an
abelian transcendental semigroup S, then I(S) = I(T), J(S) = J(T)and F(S) =
F(T).
In section 2, we also define another notion of index which is called Rees index.
We also proved that if subsemigroup Thas finite Rees index in semigroup S, then
I(S) = I(T), J(S) = J(T) and F(S) = F(T).
From [14, Theorem 3.1 (1) and (3)], we can say that Fatou set and escaping
set of holomorphic semigroup may be empty. The result [11, Theorems 5.1] is one
of the case of non-empty Fatou set and that of [16, Theorems 3.3] is a case of
the non-empty escaping set of transcendental semigroup. We see another case of
non-empty Fatou set and escaping set on the basis of the following definition.
Definition 1.3 (Partial fundamental set and fundamental set).A set U
is a partial fundamental set for the semigroup Sif
(1) U6=∅,
(2) U⊂R(S),
(3) f(U)∩U=∅for all f∈S.
If in addition to (1),(2) and (3) Usatisfies the property
(4) Sf∈Sf(U) = R(S),
then Uis called fundamental set for S.
The set R(S) is defined and discussed in section 4 of remark 4.1. From the
statements F(S)⊂Tf∈SF(f) and I(S)⊂Tf∈SI(f) ([14, Theorem 3.1]), we can
say that the Fatou set and the escaping set of holomorphic semigroup may be empty.
On the basis of the definition 1.3, we prove the following result.
Theorem 1.2.Let Sbe holomorphic semigroup and Uis a partial fundamental
set for S, then U⊂F(S). If, in addition, Sbe transcendental semigroup and Uis
a fundamental set, then U⊂I(S).
The organization of this paper is as follows: In section 2, we briefly review no-
tion of finite index and co-finite index with suitable examples and we review some
results from rational (sub) semigroup dynamics and we extend the same in tran-
scendental (sub) semigroup dynamics. We introduce Rees index of subsemigroup
and we prove the dynamical similarity of holomorphic semigroup and its subsemi-
group. In section 3, we prove theorem 1.1 and we also prove theorem1.1 by loosing
the condition of abelian if the subsemigroup has finite Rees index. In section 4, we
define discontinuous transcendental semigroup and on the basis of this notion, we
FATOU SET, JULIA SET AND ESCAPING SET IN HOLOMORPHIC SUB... 5
discuss partial fundamental set and fundamental sets and then we prove theorem
1.2.
2. Results from general holomorphic (sub) semigroup dynamics
There are various notions of investigation of how large a substructure is inside
of an algebraic object in the sense of sharing properties and structures. One of
the such a notion is index and it is an outstanding idea in general group theory
and semigroup theory. It occurs in many important theorems of group theory and
semigroup theory. The notion of finite index, cofinite index and Rees index of
subsemigroup was used to compare how much the size of subsemigroup is large
enough in semigroup. If the subsemigroup Tis big enough in semigroup S, then S
and Tshare many properties. In this context, our proposed theorem 1.1 states that
if Thas finite index or cofinte index in S, then both Sand Tshare the same Fatou
set, Julia set and escaping set. In semigroup theory, cofinite index is also known as
Grigorochuk index and this index was introduced by Grigorochuk [3] in 1988. Note
that Tis cofinite subsemigroup of a semigroup Sif it has a cofinite index in S.
Maltcev and Ruskuc [10, Theorem 3.1] proved that if for every f∈Sof a finitely
generated semigroup and every proper cofinite subsemigroup T, then f◦T6=S.
Note that if semigroup were a group, the notion of finite index and cofinite index
coincide. The subsemigroup Tof a finitely generated semigroup Sconsisting of all
words of finite (some multiple of integer n) length (compositions of finite number
of holomorphic functions) has finite index and cofinite index in S. For instance,
for any holomorphic function f, the subsemigroup hfnialways has finite index and
cofinite index in a semigroup hfi.
We first see an alternative form of finite index and cofinite index of any sub-
semigroup of holomorphic semigroup. Let Tbe a subsemigroup of holomorphic
semigroup S. For any f∈S1where S1=S∪ {Identity}, the set of form f◦T( or
T◦f) is called translate of Tby the function f. Let us define following two types
of indexes:
(1) The transnational index of Tin Sis the number of distinct translates f◦T
of Tin S.
(2) The strong orbit transnational index of Tin Sis the number of distinct
translates f◦Tof Tin Ssuch that f◦g◦T=Tfor some g∈S1.
We prove the following result which show that finite index and transnational index
are equivalent and cofinite index and strong orbit transnational index are equivalent.
Theorem 2.1.Let Tbe a subsemigroup of holomorphic semigroup S. Then
(1) Thas finite index in Sif and only if it has transnational index.
(2) Thas cofinite index in Sif and only if it has strong orbit transnational
index.
Proof. The proof is clear from definitions.
From this theorem 2.1 and definition 1.2, the finite index and cofinite index of
subsemigroups of the following examples will be clear.
6 B. H. SUBEDI AND A. SINGH
Example 2.1.[4, Example in page 362] The subset T=hf2, g2, f ◦g, g ◦fiof
semigroup S=hf, giis a subsemigroup of Sand it has finite index 3 and cofinite
index 2 in S.
The more concrete example but similar to the above example 2.1 is as follows.
The semigroup S=hsin z, cos zihas a subsemigroup
T=hsin sin z,cos cos z,sin cos z,cos sin zi
Let us denote f1=id, f2= sin zand f3= cos z. Then for any f∈S, we have
f=fi◦h∈fi◦Tfor some h∈T. The number distinct translates of Tin Sare
f1◦T=T, f2◦Tand f3◦T. So S=T∪f2◦T∪f3◦T. This shows that Thas
finite index 3 in S.
Furthermore, if f∈S, then f=hor f= sin z◦hor f= cos z◦hfor h∈T. Let
us choose f1=id and f2= sin zor cos z. If f=h, then f1◦h=id ◦h=h∈T. If
f= sin z◦h, then f2◦f= sin z◦sin z◦h= sin sin z ◦h∈T. If f= cos z◦h, then
f2◦f= cos z◦sin z◦h= cos sin z ◦h∈T. We can choose other combinations, but
anyway, we get element of semigroup T. This shows that cofinite index of Tin S
is 2.
Example 2.2.Let S=hf, giand T={words (composition) begining with f}.
Thas no finite index in S. The only cofinite subsemigroup of Tis Titself. So T
has cofinite index 1 in S. Note that Sfinitely generated but Tis not. Since any
generating set of Tmust contain {f◦gn:n>1}.
Example 2.3.Let S=hfiwhere fis a holomorphic map and T=hfni. The
subsemigroup Thas n-different translates in S, which are T, f ◦T,...,fn−1◦T. So
Thas finite index n in S. In this case, the only cofinite subsemigroup of Tis T
itself. So Thas cofinite index 1 in S.
In example 2.3, if we choose subsemigroup of Sas a Sitself, then there are
infinitely many translates of S, namely, h◦S=h◦ hfifor all h∈S. So, Shas no
finite index in itself. Again, it has cofinite index 1 in itself.
From the theorem 3.1 of [14], we can prove the following result:
Lemma 2.1.For any subsemigroup Tof a holomorphic semigroup S, we have
F(S)⊂F(T), J(S)⊃J(T).
Proof. We prove F(S)⊂F(T). The next one J(T)⊃J(S) will be proved
by taking the complement of F(S)⊂F(T) in C. By the theorem 3.1 of [14],
F(S)⊂ ∩f∈SF(f) and F(T)⊂ ∩g∈TF(g) for any subsemigroup Tof semigroup
S. Since any g∈Tis also in S, so by the same theorem 3.1 of [14], we also have
F(S)⊂F(g) for all g∈Tand hence F(S)⊂ ∩g∈TF(g). Now for any z∈F(S), we
have z∈ ∩g∈TF(g) for all g∈T. This implies z∈F(g) for all g∈T. This proves
z∈F(T) and hence F(S)⊂F(T).
Hinkannen and Martin [4, Theorem 2.4] proved that if a subsemigroup Thas
finite index or cofinite index in the rational semigroup S, then F(S) = F(T) and
J(S) = J(T). In the following theorem, we prove the same result in the case of
FATOU SET, JULIA SET AND ESCAPING SET IN HOLOMORPHIC SUB... 7
general holomorphic semigroup. Note that in our study, by general holomorphic
semigroup, we mean either it is rational semigroup or a transcendental semigroup.
Theorem 2.2.If a subsemigroup Thas finite index or cofinite index in the
holomorphic semigroup S, then F(S) = F(T)and J(S) = J(T).
Sketch of the proof. From the above lemma 2.1, F(S)⊂F(T) for any
holomorphic semigroup S. If Sis a rational semigroup, the result follows from [4,
Theorem 2.4]. We prove other inclusion if Sis a transcendental semigroup.
Let subsemigroup Tof a semigroup Shas finite index n, then by the definition
1.2, there exists finite collection of elements {f1, f2,...,fn}of S∪ {Identity}such
that
S=f1◦T∪f2◦T∪...∪fn◦T
Then for any g∈S, there is h∈Tsuch that g=fi◦h. Choose a sequence (gj)j∈N
in S, then each gjis of the form gj=fi◦hj, where hj∈T, 1 6i6n. Here, we may
assume same ifor all j. So without loss of generality, we may choose a subsequence
(gjk) of (gj) such that gjk=fi◦hjkfor particular fi, where (hjk) is a subsequence
of (hj) in T. Since on F(T), the sequence (hjk) has a convergent subsequence so do
the sequences (gjk) and (gj) in F(S). This proves F(T)⊂F(S).
Let subsemigroup Tof a semigroup Shas cofinite index n, then by the definition
1.2, there exists finite collection of elements {f1, f2,...,fn}of S∪ {Identity}such
that for every f∈S, there is i∈ {1,2,...,n}such that fi◦f∈T. Let us choose
a sequence (gj)j∈Nin S, then for each j, there is a iwith 1 6i6nsuch that
fi◦gj=hj∈T. Let z∈F(T). Then the sequence hjhas convergent subsequence
in Tso does the sequence (gj) in F(S). This proves F(T)⊂F(S).
Next, we see a special subsemigroup of holomorphic semigroup that yields cofi-
nite index.
Definition 2.1 (Stablizer, wandering component and stable domains).
For a holomorphic semigroup S, let Ube a component of the Fatou set F(S)and
Ufbe a component of Fatou set containing f(U)for some f∈S. The set of the
form
SU={f∈S:Uf=U}
is called stabilizer of Uon S. If SUis non-empty, we say that a component U
satisfying Uf=Uis called stable basin for S. The component Uof F(S)is called
wandering if the set {Uf:f∈S}contains infinitely many elements. That is, Uis
a wandering domain if there is sequence of elements {fi}of Ssuch that Ufi6=Ufj
for i6=j. Furthermore, the component Uof F(S)is called strictly wandering if
Uf=Ugimplies f=g. A stable basin Uof a holomorphic semigroup Sis
(1) attracting if it is a subdomain of attracting basin of each f∈SU
(2) supper attracting if it is a subdomain of supper attracting basin of each
f∈SU
(3) parabolic if it is a subdomain of parabolic basin of each f∈SU
(4) Siegel if it is a subdomain of Siegel disk of each f∈SU
8 B. H. SUBEDI AND A. SINGH
(5) Baker if it is a subdomain of Baker domain of each f∈SU
(6) Hermann if it is a subdomain of Hermann ring of each f∈SU
In classical holomorphic iteration theory, the stable basin is one of the above
types but in transcendental iteration theory, the stable basin is not a Hermann
because a transcendental entire function does not have Hermann ring [5, Proposition
4.2].
Note that for any rational function f, we always have Uf=U. So USis non-
empty for a rational semigroup S. However, if fis transcendental, it is possible that
Uf6=U. So, USmay be empty for transcendental semigroup S. Bergweiler and
Rohde [1] proved that Uf−Ucontains at most one point which is an asymptotic
value of fif fis an entire function. Note that value in Uf−Uneed not be omitted
value. For example, the transcendental entire function f(z) = ze−(1/2z2+3/2z−1) has
an attracting fixed point 0. Since f(z)→0 as n→ ∞, so 0 is an asymptotic value
of f. If we let Ua component of Fatou set F(f) that contains all large positive
real numbers, then 0 /∈f(U). There is a Fatou component Ufcontaining f(U) that
contains 0.
Lemma 2.2.Let Sbe a holomorphic semigroup. Then the stabilizer SU(if it is
non-empty) is a subsemigroup of Sand F(S)⊂F(SU), J(S)⊃J(SU).
Proof. Let f, g ∈SU, then by the definition 2.1, Uf=Uand Ug=U. Since Uf
and Ugare components of Fatou set containing f(U) and g(U) respectively. That
is, f(U)⊆Uf=Uand g(U)⊆Ug=U=⇒(f◦g)(U) = f(g(U)) ⊆f(Ug) =
f(U)⊆Uf=U. Since (f◦g)(U)⊆Uf◦g. Then either Uf◦g⊆Uor U⊆Uf◦g. The
only possibility in this case is Uf◦g=U. Hence f◦g∈SU, which proves that SU
is a subsemigroup of S. The proof of F(S)⊂F(SU), J(S)⊃J(SU) follows from
lemma 2.1.
There may be a connection between no wandering domains and the stable basins
of cofinite index. We have proposed the connection in the following statement for
general holomorphic semigroup S.
Theorem 2.3.Let Sbe a holomorphic semigroup with no wandering domains.
Let Ube any component of Fatou set. Then the forward orbit {Uf:f∈S}of U
under Scontains a stabilizer of Uof cofinite index.
Proof. If Sis a rational semigroup, the proof see for instance in [4, Theorem
6.1].
If Sis a transcendental semigroup, we sketch our proof in the following way.
We have given that Ube a non-wandering component of Fatou set F(S). So U
has a finite forward orbit U1, U2,...,Un(say) with U1=U.
Case (i): If for every i= 1,2,...n, there is fi∈Ssuch that fi(Ui)⊆U1, then by
the above lemma 2.2, stabilizer SU1={f∈S:U1f=U1}is a subsemigroup of
S. For any f∈Sthere is fifor each i= 1,2,...,n such that U1fi◦f=U1. Which
shows that fi◦f∈SU1. Therefore, U1is a required stable basin such that the
stabilizer SU1has cofinite index in S.
FATOU SET, JULIA SET AND ESCAPING SET IN HOLOMORPHIC SUB... 9
Case (ii): If for every j= 2,...n, there is fj∈Ssuch that fj(Uj)⊆V, where
V=Ujsuch that j>2, then number of components of forward orbits of Vis
strictly less than U. By this way, we can find a component W=Uifor some
16i6nwhose forward orbit has fewest components. For every component Wg
of the forward orbit of W, there is a f∈Ssuch that f(Wg)⊆W. That is,
Wg◦f=W, and it follows that Wis a required stable basin such that the stabilizer
SWhas cofinite index.
Note that in our forth coming study, the subsemigroup SUof cofinite index in
Sis replaced by a basin Uof cofinite index. In this sense, above theorem 2.3 is
a criterion to have a basin Uof cofinite index. Similar to rational semigroups [4,
Conjectures 6.1 and 6.2], we have hoped that the following analogous two statements
will also be conjectures in the case of transcendental semigroup S. Note that each
is true in classical complex dynamics (rational and transcendental).
Conjecture 2.1.Let Sbe a (finitely generated) transcendental semigroup
such that F(S)6=∅. Then a stable basin Uhas cofinite index in S.
Conjecture 2.2.Let Sbe a finitely generated transcendental semigroup.
Then for each component Uof F(S), there is a stable basin Vfor Slying in
the forward orbit of Uhas cofinite index in S.
Note that there are examples of holomorphic semigroups whose subsemigroups
have cofinite index but not have finite index. For example, for any f∈S(holomor-
phic semigroup), the sets S◦f={g◦f:g∈S}and f◦S={f◦g:g∈S}are
subsemigroups (in fact, left and right ideals (see for instance in [16, Definition 2.2
and Proposition 2.1])) of S. Each of these subsemigroups has cofinite index 1 in S
but not have finite index in S.There may be a lot of subsemigroups of holomorphic
semigroup Sthat may have finite index in S. If we able to find such subsemigroups,
there will be a chance of replacing cofinite index by finite index and so further inves-
tigations will be more interesting. We leave this notion of investigation for future
research.
From the example just we mentioned in above paragraph, we can also say that
S◦fand f◦Sare not finitely generated even if the semigroup Sis. For if
S◦f=hf1, f2, . . . , fniwhere fi∈Sfor i= 1,2,...n, then fi=gi◦f, where
gi∈S. For any g∈S, we have gn◦f∈S◦ffor all n>1 but not every
gn◦f∈ hf1, f2, . . . , fni. This fact together with discussion in above paragraph,
we came to know that the notion of cofinite index fails to preserve basic finiteness
(finitely generated) condition of subsemigroup. That is, if Tis a subsemigroup of
cofinite index in semigroup S, then Sis finitely generated may not always imply
Tis finitely generated. There is another notion of index which preserves finiteness
condition of subsemigroup.
Definition 2.2 (Rees index).Let Sbe a semigroup and Tbe its subsemigroup.
The Rees index of Tin Sis defined as |S−T|+ 1. In this case, we say Tis large
subsemigroup of Sand Sis small extensions of T.
10 B. H. SUBEDI AND A. SINGH
This definition was first introduced by A. Jura [6] in the case when Tis an
ideal of semigroup S. In such a case, the Rees index of Tin Sis the cardinality of
factor semigroup S/T . From the definition 2.2, it is clear that the Rees index of T
in Sis the size of the compliment S−T. To have Rees index of any subsemigroup
in its parent semigroup is fairly a restrictive property, and it occurs naturally in
semigroups ( for instance all ideals in additive semigroup of positive integers are of
finite Rees index). Note that Rees index does not generalize group index, even the
notion of finite Rees index does not generalize finite group index. That is, if Gis
an infinite group and His proper subgroup, then group index of Hin Gis finite
but Rees index is infinite. In fact, let Gbe an infinite group and His a subgroup
of G, then Hhas finite Rees index in Gif and only if H=G.
Next, we investigate how much similar a semigroup Sand its large subsemigroup
Tare. One of the basic similarity (proved first by Jura [6]) is the following result.
Theorem 2.4.Let Tbe a large subsemigroup of semigroup S. Then Sis finitely
generated if and only if Tis.
Proof. See for instance [13, Theorem 1.1].
On the basis of this theorem 2.4, we proof the following dynamical similarity of
holomorphic semigroup and its subsemigroup.
Theorem 2.5.Let Tbe a large subsemigroup of finitely generated holomorphic
semigroup S. Then F(S) = F(T)and J(S) = J(T).
Proof. We prove F(S) = F(T), another one is clear by taking compliment.
By the lemma 2.1, it is clear that F(S)⊂F(T). So, we prove only F(T)⊂F(S).
By above theorem 2.4, Tis finitely generated and let X={f1, f2,...,fn} ⊂ Sbe a
generating set of T. Then clearly Sis generated by the set Y=X∪(S−T). Every
sequence (fi) in F(T) where fi=fi1◦fi2◦...◦finhas a convergent subsequence.
Now each element gmof a sequence (gm) in Scan be written as gm=fi1◦fi2◦
...◦fin◦hj1◦hj2◦...◦hjk, where S−T={h1, h2,...,hk} ⊂ S. Since S−Tis
finite, so convergent sequence in F(T) can be finitely extended to the convergent
sequence in F(S). So, every sequence (gm) in F(S) has a convergent subsequence.
Hence F(T)⊂F(S).
3. Proof of the Theorem 1.1
In this section, we concentrate on our mission of proving theorem 1.1. We first
prove the analogous result of lemma 2.1 in the case of escaping set of transcendental
semigroup.
Lemma 3.1.For any subsemigroup Tof a transcendental semigroup S, we have
I(S)⊂I(T).
Proof. By the theorem 3.1 of [14], I(S)⊂ ∩f∈SI(f) and I(T)⊂ ∩g∈TI(g)
for any subsemigroup Tof semigroup S. Since any g∈Tis also in S, so by
the same theorem 3.1 of [14], we also have I(S)⊂I(g) for all g∈Tand hence
I(S)⊂ ∩g∈TI(g). Now for any z∈I(S), we have z∈ ∩g∈TI(g) for all g∈T. This
FATOU SET, JULIA SET AND ESCAPING SET IN HOLOMORPHIC SUB... 11
implies z∈I(g) for all g∈T. By the definition 1.1, we have gn(z)→ ∞ as n→ ∞
for all g∈T. This proves z∈I(T) and hence I(S)⊂I(T).
Lemma 3.2.Let Sbe a transcendental semigroup. Then
(1) int(I(S)) ⊂F(S)and ext(I(S)) ⊂F(S), where int and ext respectively
denote the interior and exterior of I(S).
(2) ∂I (S) = J(S), where ∂I(S)denotes the boundary of I(S).
Proof. We refer for instance lemma 4.2 and theorem 4.3 of [8].
Note that this lemma 3.2 is a extension of Eremenko’s result [2], ∂I(f) = J(f)
of classical transcendental dynamics to more general semigroup settings.
Proof of the Theorem 1.1. We prove I(S) = I(T). The proof of J(S) =
J(T) is obvious from above lemma 3.2 (2). The fact F(S) = F(T) is also obvious.
By the above lemma 2.1, we always have I(S)⊂I(T) for any subsemigroup Tof
semigroup S. For proving this theorem it is enough to show the opposite inclusion
I(T)⊂I(S).
Let subsemigroup Tof a semigroup Shas finite index n, then by the definition
1.2, there exists finite collection of elements {f1, f2,...,fn}of S∪ {Identity}such
that
S=f1◦T∪f2◦T∪...∪fn◦T
Then for any g∈S, there is h∈Tsuch that g=fi◦h. Choose a sequence (gj)j∈N
in S, then each gjis of the form gj=fi◦hj, where hj∈T, 1 6i6n. Here we may
assume same ifor all j. Let z∈I(T), then by [14, Theorem 2.2], every sequence
(hj)j∈Nin Thas a divergent subsequence (hjk)jk∈N. That is, hn
jk(z)→ ∞ as n→ ∞
for all jk. In this case, every sequence (gj)j∈Nin Shave subsequence (gjk)k∈N, where
gjk=fi◦hjkwith hn
jk(z)→ ∞ as n→ ∞. Since Sis an abelian transcendental
semigroup, so gjk=fi◦hjk=hjk◦fi. Thus we may write gn
jk(z) = hn
jk(fi(z)) → ∞
as n→ ∞. This shows that fi(z)∈I(S). If fi= identity for particular i, we are
done. If fiis not identity, then of course, it is an element of abelian transcendental
semigroup Sand in this case, I(S) is backward invariant by [16, Theorem 2.6]. So
we must have z∈I(S). Therefore, I(T)⊂I(S).
Let subsemigroup Tof a semigroup Shas cofinite index n, then by the definition
1.2, there exists finite collection of elements {f1, f2,...,fn}of S∪ {Identity}such
that for every f∈S, there is i∈ {1,2,...,n}such that fi◦f∈T. Let us choose
a sequence (gj)j∈Nin S, then for each j, there is a iwith 1 6i6nsuch that
fi◦gj=hj∈T. Let z∈I(T), then by the same argument stated in the first
part, every sequence (hj)j∈Nin Thas a divergent subsequence (hjk)jk∈Nat point z.
This follows that sequence (fi◦gj) has a divergent subsequence (fi◦gjk) (say) at z.
Since Sis abelian, so we can write (fi◦gjk)(z) = (gjk◦fi)(z) = gjk(fi(z)) = hjk(z).
Now for any z∈I(T), hjk∈T, we must have hn
jk(z) = gn
jk(fi(z)) → ∞ as n→ ∞.
This implies that fi(z)∈I(S). If fi= identity for particular i, we are done. If fiis
not an identity, then of course, it is an element of abelian transcendental semigroup
12 B. H. SUBEDI AND A. SINGH
S, then by the same fact explained in the last part of above paragraph, we have
I(T)⊂I(S).
The condition of abelian can be loosed from the theorem 1.1 if we choose Rees
index. So we can generalize this theorem to the following result.
Theorem 3.1.If subsemigroup Tof a finitely generated transcendental semi-
group Shas finite Rees index, then I(S) = I(T), J (S) = J(T)and F(S) = F(T).
Proof. The last two, that is, J(S) = J(T) and F(S) = F(T) was proved in
theorem 2.5 of section 2. Also, if we prove I(S) = I(T), then J(S) = J(T) is
obvious from above lemma 3.2 (2). The fact I(S)⊂I(T) for any subsemigroup T
of semigroup Sis obvious. So we prove I(T)⊂I(S).
By the theorem 2.4, Tis finitely generated and let X={f1, f2,...,fn} ⊂ Sbe a
generating set of T. Then clearly Sis generated by the set Y=X∪(S−T). By [14,
Theorem 2.2], every sequence (fi) in Twhere fi=fi1◦fi2◦...◦finfor each 1 6i6n
has a divergence subsequence (fnk) at each point of I(T). Now each element gmof a
sequence (gm) in Scan be written as gm=fi1◦fi2◦...◦fin◦hj1◦hj2◦...◦hjk, where
S−T={h1, h2,...,hk} ⊂ Sis a finite set. This show that divergent sequence
in I(T) can be extended finitely to divergent sequence in I(S). So, every sequence
(gm) in I(S) has a divergent subsequence. Hence I(T)⊂I(S).
4. Proof of the Theorem 1.2
It is known to us that for certain holomorphic semigroups, the Fatou set and
the escaping set might be empty. In this section, we discuss notion of discontinuous
semigroup. Such type of notion yields partial fundamental set and fundamental set.
We prove theorem 1.2 by showing partial fundamental set is in Fatou set F(S) and
fundamental set is in escaping set I(S).
Definition 4.1 (Discontinuous semigroup).A semigroup Sis said to be
discontinuous at a point z∈Cif there is a neighborhood Uof zsuch that f(U)∩
U=∅for all f∈Sor equivalently, translates of Uby distinct elements of S
(S-translates) are disjoint. The neighborhood Uis also called nice neighborhood of
z.
Remark 4.1.Given a holomorphic semigroup S, there are two natural subsets
associated with S.
(1) The regular set R(S) that consists of points z∈Cat which Sis discontin-
uous.
(2) The limit set L(S) that consists of points z∈Cfor which there is a point
z0, and a sequence {fn}of distinct elements of Ssuch that fn(z0)→zas
n→ ∞.
A set X⊂Cis S-invariant or invariant under Sif f(X) = Xfor all f∈S.
It is clear that both of the sets R(S) and L(S) are S-invariant. If Uis a nice
neighborhood, then every point of Ulies in R(S). So R(S) is a open set where as
the set L(S) the a close set and R(S)∩L(S) = ∅. Recall that a set Uis a partial
FATOU SET, JULIA SET AND ESCAPING SET IN HOLOMORPHIC SUB... 13
fundamental set for the semigroup Sif (1) U6=∅, (2) U⊂R(S), (3) f(U)∩U=∅
for all f∈S. If in addition to (1), (2) and (3), U satisfies the property (4)
Sf∈Sf(U) = R(S), then Uis called fundamental set for S. We say that x, y ∈C
are S- equivalent if there is a f∈Ssuch that f(x) = y. Above condition (3) asserts
that any two points of Uare not S-equivalent under semigroup S, and condition
(4) asserts that every point of R(S) is equivalent to some point of U. Note that if
we replace (3) by f−1(U)∩U=∅for all f∈S, we say Uis a backward partial
fundamental set for Sand if, in addition with condition (4), we say Uis backward
partial fundamental set. Note that the following two theorems 1.2 and 4.1 hold if
we have given partial backward fundamental set in the statements. Similar to the
results of Hinkkanen and Martin [4, Lemma 2.2] in the case of rational semigroup,
we have prove the following in the case of transcendental semigroup S.
Proof of the Theorem 1.2. Let Sis a holomorphic semigroup. Uis non-
empty open set and f(U)∩U=∅for all f∈Sby definition 4.1. The statement
f(U)∩U=∅for all f∈Simplies that Somits Uon U. Since Uis open, it
contains more than two points. Then by Montel’s theorem, Sis normal on U. So,
U⊂F(S).
Let Sis a transcendental semigroup.To prove U⊂I(S), we have to show that
fn(z)→ ∞ as n→ ∞ for all f∈Sand for all z∈U. The condition f(U)∩U=∅
for all f∈Simplies that fn(U)∩U=∅because as f∈S, then so fn∈S. Also,
Uis a fundamental set, so by the definition 1.3 (4), we have Sf∈Sf(U) = R(S).
So by the remark 4.1(2), there are no points in Uwhich appear as the limit points
under distinct (fm)m∈Nin S. That is, (fm) has divergent subsequence (fmk) at each
point of U. Thus, by [14, Theorem 2.2], for any z∈U, fn(z)→ ∞ as n→ ∞ for
any f∈(fm). This shows that U⊆I(S).
Finally, we try to generalize above result (Theorem 1.2) in the following form.
We have given here a short sketch of the proof. For more detail proof, we refer [12,
Theorem 2.1].
Theorem 4.1.Let U1and U2be two (partial) fundamental sets for transcenden-
tal semigroups S1and S2respectively. Suppose furthermore that C r U1⊂U2and
C r U2⊂U1. Then the semigroup S=hS1, S2iis discontinuous, and U=U1∩U2
is a (partial) fundamental set for semigroup S.
Sketch of the proof. Let U1,U2and S1,S2as given in the statement of
the theorem, it is clear from the theorem 1.2 that F(S1)6=∅, F (S2)6=∅and
(I(S1)6=∅, I(S2)6=∅if U1and U2are fundamental sets of S1and S2respectively).
Note that U6=∅by the assumption of the theorem and f(U)∩U=∅for every
f∈Sfollows easily. This proves Sis discontinuous together with Uis a (partial)
fundamental set for S.
14 B. H. SUBEDI AND A. SINGH
References
[1] Bergweiler, W. and Rohde, S.: Omitted value in the domains of normality, Proc. Amer. Math.
Soc. 123 (1995), 1857-1858.
[2] Eremenko, A.: On the iterations of entire functions, Dynamical System and Ergodic Theory,
Banach Center Publication Volume 23, Warsaw, Poland, (1989).
[3] Grigorochuk, R. I.: Semigroups with cancellations of degree growth, Mat. Zamekti 43, no. 3
(1988), 305-319.
[4] Hinkkanen, A. and Martin, G.J.: The dynamics of semi-groups of rational functions- I, Proc.
London Math. Soc. (3) 73, 358-384, (1996).
[5] Hua, X.H. and Yang, C.C.: Dynamic of transcendental functions, Gordon and Breach Science
Publication, (1998).
[6] Jura, A.: Determining ideals of a given finite index in a finitely presented semigroup, Demon-
stratio Math. 11 (1978), 813-827.
[7] Kumar, D.: On escaping set of some families of entire functions and dynamics of composite
entire functions, arXiv: 1406.2453v4 [math. DS] 4 Oct. 2015.
[8] Kumar, D. and Kumar, S.: The dynamics of semigroups of transcendental entire functions-II,
arXiv: 1401.0425 v3 (math.DS), May 22, 2014.
[9] Kumar, D. and Kumar, S.: Escaping set and Julia set of transcendental semigroups,
arXiv:141.2747 v3 (math. DS) October 10, 2014.
[10] Maltcev, V. and Ruskuc, M.: On hopfian cofinite subsemigroups, arXiv: 1307. 6929v1 [math.
GR], 26 Jul. 2013.
[11] Poon, K.K.: Fatou-Julia theory on transcendental semigroups, Bull. Austral. Math. Soc. Vol-
58 (1998) PP 403-410.
[12] Poon, K.K.: Fatou-Julia theory on transcendental semigroups II, Bull. Austral. Math. Soc.
Vol- 55 (1999) PP 257-262.
[13] Ruskuc, N.: On large subsemigroups and finiteness conditions of semigroups, proc. London
Math. Soc. (3) 76 (1998), 383-405.
[14] Subedi, B. H. and Singh, A.: A short comparison of classical complex dynamics and holo-
morphic semigroup dynamics, arXiv: 1803.08368v1[math. DS] 22 March 2018.
[15] Subedi, B. H. and Singh, A.: Invariant properties of Fatou set, Julia set and escaping set of
holomorphic semigroup, arXiv: 1803.09662v1[math. DS] 23 March 2018.
[16] Subedi, B. H. and Singh, A.: Escaping set of hyperbolic semigroup, arXiv:1803.10381v1 [math
DS], 28 March, 2018.
Central Department of Mathematics, Institute of Science and Technology,
Tribhuvan University, Kirtipur, Kathmandu, Nepal
E-mail address:subedi.abs@gmail.com / subedi bh@cdmathtu.edu.np
Central Department of Mathematics, Institute of Science and Technology,
Tribhuvan University, Kirtipur, Kathmandu, Nepal
E-mail address:singh.ajaya1@gmail.com / singh a@cdmathtu.edu.np
